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DESCRIPTION 

RECORDING MEDIUM , ^lAYBACK APPARATUS AND METHOD , RECORDING 

METHOD, AND COMPUTER-READABLE PROGRAM 

Technical Field 

The present invention relates to a field of user" interface 
control- More particularly, the present invention relates to 
improvements in recording mediums used for consumer movie 
distribution, as well as in consumer playback apparatuses having 
a user interface for playback of the recording mediums . 

Background Art 

A user interface using a menu structure is an essential 
function for receiving user operations, such as selection of 
languages and chapters for playback. 

When a movie offers a number of user-selectable options 
such as chapters, there may be a case where all the options can 
not be presented on a single screen display . Accordingly, there 
is a demand for a multi-page menu with which multiple pages are 
sequentially presented in response to user operations, so that 
numerous options are appropriately presented on respective pages 
of the menu. 

Unfortunately, however, if too many pages are displayed, 
those pages occupy a large portion of the screen, thereby 
interfering with viewing the main movie also displayed on the 
screen. Such a problem can be avoided if a user repeatedly makes 
operations for removing each page that is no longer of his interest 



WO 2005/048261 PCT/JP2004/017215 

However, the need to make such troublesome operations over and 
over may be another factor bothering the user watching the movie . 

In addition, the interactive control that a DVD provides 
has the following disadvantage. For each menu page to be 
5 displayed, an access needs to be made to the DVD to read page 
information. Because of the access, the menu page can not be 
displayed without interruption to video playback. The 
interruption makes it difficult for the user to concentrate on, 
and enjoy the movie. 

10 

Disclosure of the Invention 

The present invention aims to provide a recording medium 
supporting presentation of multi-page menus which will not 
interfere with the video playback. 

15 ' To achieve the above aim, the present invention provides 

a recording medium having a video stream and a graphics stream 
recorded thereon. The video stream represents a moving picture 
made up of a plurality of pictures. The graphic stream is used 
for overlaying a multi-page menu on the moving picture, and 

20 includes interactive control- information and graphics data. 
The graphics data is used for generating the multi-page menu. 
The interactive control information includes time information 
used for controlling behavior of the multi-page menu in 
accordance with a playback proceeding of the video stream. 

25 According to the recordingmediumhaving the above structure , 

the interactive control information multiplexed in the graphics 
stream includes control information. The control information 
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defines the multi-page, menu behavior in accordance with the 
proceeding of the movie playback. This structure makes it 
possible to automatically present a number of pages during the 
time the movie playback is showing scenery , for example. In 
5 addition, it is also possible to automatically remove the pages 
from the screen when the movie playback reaches a climax scene. 
With such control, the multi-page menu is presented without 
interfering with viewing the movie playback. Consequently, the 
user can en' joy the convenience provided by the multi-page menus, 

10 while watching the movie playback without obstructive display. 

In addition, the information used for controlling" the 
multi-page menu behavior is included in the graphics stream-, 
and the graphics stream is multiplexed with the video stream. 
This structure allows a playback apparatus to read the 

15 information needed for presentation of the multi-page menu, 
without interrupting the video stream reading. Thus, pages of 
the multi-page menu are smoothly presented in synchronization 
with the video frame rate. 

20 Brief Description Of The Drawings 

FIG. 1 is a view showing a usage pattern of a recording 

AS 

medium according to the present invention; 

FIG. 2 is a view showing an example structure of a BD-ROM; 
FIG. 3 is a view schematically showing the structure of 

« 

25 aw AV Clip; 

■ FIG. 4A is a view showing the structure of an interactive 
graphics' stream; 

3 
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FIG - 4B is a view showing PES packets carrying functional 
segments; 

FIG . 5 is a view showing the logical structure defined by 
- various types of functional segments; 
5 FIG. 6 is a view showing the time axis of AV clip playback 

to which a DS n is allocated; 

FIGs. 7A and 7B are views showing the relations between 
interact ive_composit ion structures and ICSs; 

FIG. 8 is a view showing the internal structure of an ICS; 
10 FIG. 9 is a view showing the. processing performed on an 

ICS in the case where a stream_model field shows a "Multiplexed" 
type; 

FIG. 10 is a view showing the processing performed on an 
ICS in the case where the stream_model field shows a "Preload" 
15 type; 

FIG. 11 is a timing chart illustrating the functions of 
s elect ion_time_out_pts and composition_time_out__pts fields in 
an ICS belonging to the DS n in the case where the DS n is an Epoch 
Start DS (n = 1) ; 
20 FIG. 12A is a view showing the state transition of a 

Multi-page menu when a Pop-Up U/I is used; 

* 

FIG. 12B is a view showing the state transition of the 
Multi-page menu when an Always-On U/I is used; 

FIG . 13 is the display scenario incorporating theMulti-Page 

25 menu; 

FIGs. 14A-14C show a display example defined by the 
selection time out pts; 
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FIGs . 15A-15D show a display example defined by the 
use r_t ime_jDut_dur a t ion; 

FIG. 16 is a view showing the internal structure of page 
information associated with an arbitrary one of multiple pages 
(x th page) of the Multi-Page Menu; 

FIG. 17 is a view showing the state transition of buttons 
O-A, 0-B, O-C, and O-D; 

FIG. 18 is a view showing an example of the button_info 
description defining the button state transition shown in FIG. 
17; • 

FIG. 19 is a view showing graphical images derived from 
ODSs 11-19; 

FIG. 20 is a view showing an effect_sequence structure; 

FIG. 21A is a view schematically illustrating how ah 
ef f ect_duration is determined; 

FIG. 21.B shows an eguation for calculating the 
ef f ect_duration; 

FIG. 22 is a view showing a specific example of an in-ef feet 
animation; 

FIG. 23 is a view showing an out-effect animation presented 
in accordance with the window_width, window_height , and 
ef f ect__duration field values; 

FIG. 24 is a view showing the internal structure of an 
arbitrary composition_obj ect (i) in detail; 

FIG. 25 is a view showing a specific example of an in-ef feet 
presenting a Right-Scroll animation effect; 

FIG. 2 6 is a view- showing an example of the 
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compos it ion_obj ect ( 0 ) structure realizing the in-effect shown 
in FIG. 25; 

FIG. 27 is a view showing an example of the 
composition_obj ect (1) structure realizing the in-effect shown 
in FIG. 25; 

FIG. 28 is a view showing an example of the 
composition_obj ect (2 ) structure realizing the in-effect shown 
in FIG. 25; 

FIG. 2 9 is a view showing an example of the 
composition_obj ect (3) . structure realizing the in-effect shown 
in FIG. 25; 

FIG. 30 is a view showing the order of ODSs within the DS n ; 

FIG. 31 is a view showing how the Order of ODSs within S-ODSs 
differs between the cases where the 

def ault_selected_button_id_ref field is set to xx 00" and to the 
valid value specifying a button B; 

FIG. 32A is a view showing a LASTODSPTS in the case where 
the default selected button is statically determined; 

FIG. 32B is a view showing a LASTODSPTS in the case where 
the default selected button is dynamically determined; 

FIG. 33A is a view showing a LASTODSPTS in the case where 
the DS includes no ODS associated with the selected state; 

FIG. 33B is a view showing a LASTODSPTS in the case where 
the DS includes no ODS associated with the normal state; 

FIG. 33C is a view showing a LASTODSPTS in the case where 
the DS includes no ODS at all; 

FIG. 34A is a view showing an algorithm for obtaining the 

6 
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DECODEDURATION value from the value calculated by 
LASTODSPTS (DS n ) - DTS ( DS n [ ICS ] ) and the value of PLANECLERATIME; 

FIGs. 34B and 34C are views showing' examples of how to 
calculate the DECODEDURATION value; 
5 FIG. 35A is a view showing an equation for calculating the 

EFFECTTD; 

FIG. 35B is a view showing an equation for calculating the 
PAGDEFAULTTD in the case where no in-ef f ect is provided and the 
default selected button is statically determined; 
10 FIG. 35C is a view showing an equation for calculating the 

PAGENODEFAULTTD in the case where no in-ef feet is provided and 
the default selected button is dynamically determined; 

FIG. 36 is a view showing an algorithm for selectively 

determining one of EFFECTTD, PAGE DE FAULT T D , and PAGENODEFAULTTD 

« 

15 as TRANSFERDURATION ; 

FIG . 37 is a view showing the internal structure of a playback 
apparatus according to the present invention; 

FIG. 38A is a flowchart of "procedure when playback 
condition is changed", preformed with respect to a PSR 11; 
20 FIG. 38B is a flowchart of "procedure when change is 

requested", preformed with respect to the PSR 11; 

FIG. 3 9A is a flowchart of "procedure when playback 
condition is changed", performed with respect to a PSR 10; 

FIG. 3 9B is a flowchart of "procedure when change is 
25 requested", performed with respect to the PSR 10; 

FIG. 40 is a view illustrating the internal structure of 
an I-Graphics decoder 9; 
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FIG. 41 is a flowchart of the main routine of the procedure 
performed by a graphics controller 37; 

FIG. 42 is a flowchart of the procedure for the first 
presentation; 

FIG. 43 is a flowchart of the procedure for playback of 
an in-effect animation; 

FIG. 4 4 is a flowchart of the display update procedure for 
presentation buttons in animation; 

FIG. 45 is a flowchart of the procedure for executing a 
navigation command; 

FIG. 46 shows the procedure for processing a UO; 

FIG. 47 is a flowchart of the current button change 
procedure; 

FIG. 48 is a flowchart of the procedure for auto-activation 
of a button; and 

FIG. 4 9 is a flowchart of the manufacturing process of 
the BD-ROMs according to the first to fifth embodiments. 

Best Mode for Carrying Out the Invention 
(FIRST EMBODIMENT) 

Hereinafter, a description is given to an embodiment of 
a recording medium according to the present invention. First 
of all, a description is given to usage of the recording medium, 
which is one form of practicing the present invention. FIG. 
1 shows a usage pattern of the recording medium according to 
the present invention. In the figure, a BD-ROM 100 is a recording 
medium according to the present invention. . The BD-ROM 100 is 
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used for providing movies to a home theater system composed of 
a playback apparatus 200, a remote controller 300, and a 
television 400. 

This concludes the description of the usage of recording 
medium according to the present invention. Next, a description 
is given to production of the recording medium, which is another 
form of practicing the present invention. The recording medium 
can be embodied by making improvements to an application layer 
of a BD-ROM. FIG. 2 shows an example structure of the BD-ROM 
100.. 

In the figure, the BD-ROM 100 is shown on the fourth level, 
and a BD-ROM 7 s track is shown on the third level. The track 
is stretched out into a straight line in the figure, although 
the track in practice spirals outwards from the center of the 
BD-ROM. The track is composed of a lead-in area, a volume area, 
and a lead-out area. The volume area has a layer model of a 
physical layer, a file system layer, and an application layer. 
The first level shows, in a directory structure, a format of 
the application layer (application format) of the BD-ROM. 

The first level shows the format of the application layer 
(application format) . In the figure, the BD-ROM has a ROOT 
directory, and a BDMV directory under the ROOT directory, and 
a STREAM directory under the BDMV directory. 

The STREAM directory contains an AV clip and Sub clips. 
The AV clip and Sub clips are contained in files in the STREAM 
directory, such as "00001 .m2ts", "00002 .m2ts", and "00003 .m2ts" . 
Each file in the STREAM directory has a filename of a unified 
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format, M xxxxx.m2ts" where x is an arbitrary integer. 

<AV Clip Structure> 

Now, a description is given to the AV clip. FIG. 3 is a 
5 view schematically showing the structure of the AV Clip. 

In the drawing, the middle level shows the AV clip. This 
AV clip can be created as follows. The video stream made up 
of a plurality of video frames (pictures pjl, pj2, pj3, ...) and 
the audio stream made up of a plurality of audio frames both 

10 shown on the upper first level are .separately converted to PES 
packets shown on the upper second level, and further converted 
to TS packets shown on the upper third level. Likewise, the 
presentation graphics stream (hereinafter, "PG stream" ) and the 
interactive graphics stream (hereinafter, "IG stream") both 

15 shown on the lower first level are separately converted to PES 
packets shown on the lower second level, and further converted 
to TS packets shown on the lower third level. These TS packets 
of the video, audio., PG and IG streams are multiplexed to form 
the AV clip. 

20 The PG and IG streams are data in stream format used for 

compositing graphics with each picture of the video stream. An 
entity of data carried by the PG and IG streams for overlaying 
on a picture is. referred to as a "graphics object". 

The PG stream is a graphics stream used for presentation 
25 of subtitles using the graphics objects. 

The IG stream is a graphics stream used for interactive 
•presentation of the graphics objects. This concludes the 

10 
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description of the AV clip structure. Next, a description is 
given to the AV clip and the Sub clips. The Sub clips are not 

4 

multiplexed. Rather, the Sub clips are files exclusively 
containing the IG stream, audio stream, and text data. The IG 
stream stored as a Sub clip is loaded to a memory in advance 
for synchronous playback with an AV clip. This operation of 
loading a Sub clip to a memory in advance is referred to as 
"preloading". 

Next, a description is given to the IG stream. FIG. 4A 
is a view showing the structure of the IG stream. On the first 
level, a string of TS packets constituting the AV clip is shown. 
On the second level, a string of PES packets constituting the 
graphics stream is shown. The PES packets shown on the second 
level are formed by concatenating payloads of TS packets having 
a predetermined PID within the TS packet string shown on the 
first level. Note that no further description is given to the 
PG stream because it is not the gist of the present invention. 

On the third level, the structure of graphics stream is 
shown. The graphics stream is composed of functional segments 
that include an ICS (Interactive Composition Segment) , a PDS 
( Palette Definition Segment ) , an ODS (Object Definition Segment ) , 
and an END (End of Display Set Segment) . Of these functional 
segments, the ICS is a display composition segment,' whereas the 
PDS, ODS, and END are definition segments. Each functional 
segment is either in one-to-one or one-to-multiple 
correspondence with PES packets. That is, one functional 
segment is recorded on the BD-ROM 100 after being converted to 

11 
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a single PES packet , or fragmented and converted to a plurality 
of PES packets. 

Hereinafter, a description is given one by one to those 
functional segments . 
5 The ICS is a functional segment used to define an interactive 

display composition of graphics . One example of the interactive 
display composition is a Multi-Page menu. 

The ODS is a run-length encoded graphics object composed 
of a plurality of pieces of run-length data. The run-length 

10 data expresses a pixel string using a Pixel Code which shows 
a pixel value and a continuous length of the pixel value. The 
Pixel Code is an 8-bit value showing one of the values from 1 
to 255. Through the use of this Pixel Code, the run-length data 
sets arbitrary 256 pixel colors out of full color (16,777,216 

15 colors) . . 

The PDS is a functional segment storing pallet data. The 
pallet data shows combinations of Pixel Codes of 1 to 255 and 
pixel values. The pixel value used herein is made up of a red 
color difference component (Cr value) , a blue color difference 

20 component (Cb value) , a luminance component (Y value) , and a 
transparency (T value) . Substituting a Pixel Code of each piece 
of run-length data into a pixel value shown by the palette data 
produces a color. 

The END provides an explicit indication that the 

25 transmission of functional segments is complete.; The END is 
inserted into the graphics stream as the last segment. This 
concludes the description of each functional segment. 

12 
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FIG. 4B is a view showing PES packets carrying functional 
segments. As shown in the figure, each PES packet is composed 
of a packet header and a payload. The payload is an entity 
carrying a functional segment, and the packet header carries 
5 a DTS and a PTS associated with the functional segment. 
Hereinafter, a DTS and a PTS in a packet header of a PES packet 
carrying a functional segment are simply described as a DTS and 
a PTS of that functional segment. 

These various types of functional segments define a logical 

10 structure such as the one shown in FIG. 5. In the drawing, the 
first level shows Epochs, the second level shows DSs (Display 
Sets ) , and the third level shows types of the DSs . The functional 
segments shown on the third level in FIG. 4A is shown on the 
fourth level in this figure. 

15 An Epoch on the first level refers to one time unit of 

continuous memory management on a time axis of the AV clip playback, 
and also refers to a group of data allocated to that time unit. 
Memory mentioned herein includes a graphics plane for storing 
one screen of graphics objects and an object buffer for storing 

20 uncompressed graphics data . Continuous memory management means 
that neither the graphics plane nor the object buffer is flushed 
throughout the Epoch, and that clearing and rendering of graphics 
are performed only within a predetermined rectangular area on 
the graphics plane (to flush means to clear the entire graphics 

25 plane or the entire object buffer) . The size and position of 
this rectangular area are fixed throughout the Epoch. As long 
as clearing and rendering of graphics are performed within this 

13 
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fixed rectangular area on the graphics plane, seamless playback 
is guaranteed. In other words, the Epoch is a time unit of 

1 4 

seamless playback on the time axis of the AV clip playback. To 
change the graphics clearing/rendering area on the graphics plane, 
5 it is necessary to define a point on the playback time axis at 
which the change is effected and to set a new Epoch from the 
point onward. In this case, the boundary between two Epochs 
is not seamless. 

The seamless playback mentioned herein means that the 
10 clearing and rendering of graphics are each completed with a 
predetermined number of video frames . In the case of IG stream, 
the number of video frames is four to five. The number of video 
frames is determined based on the ratio of the fixed rectangular 
area to the entire -graphics plane and the transfer rate between 
15 the object buffer and the graphics plane. 

A DS on the second level is a group of functional segments 
included in the graphics stream and provides one complete display 
composition. In FIG. 5, dashed lines hkl show to which Epoch 
the DSs on the second level belong. The DSi, DS 2 , DS 3 ..., and DS n 
20 are shown to belong to the first Epoch on the first level. 

The third level shows the types of DSs. The first DS in 
an Epoch is an "Epoch Start" DS . Each DS other then the first 
one in the Epoch is either an "Acquisition Point" DS, a "Normal 
Case" DS, or an "Epoch Continue" DS . Though the DSs are shown 
25 in the order of Acquisition Point , Normal Case, and Epoch Continue, 
this order is merely an example and the DSs may be arranged in 
a different order. 
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The Epoch Start DS provides a display effect "new display", 
and indicates a start of a new Epoch. Thus, the Epoch Start 

4 

DS contains all functional segments necessary for the next screen 
composition. The Epoch Start DS is provided at a point to which 
5 a skip operation is likely to be made, such as a start of a chapter 
in a movie . 

The Acquisition Point DS provides a display effect "display 
refresh", and is identical in contents to the preceding Epoch 
Start DS . The Acquisition Point DS is not the start of the Epoch, 

10 but contains all functional segments necessary for the next 
screen composition. Therefore, graphics can be displayed 
reliably when playback is started from the Acquisition Point 
DS. That is to say, the Acquisition Point DS enables a display 
composition to be made from a midpoint in the Epoch. 

15 The Acquisition Point DS is provided at a point to which 

a skip operation may be made, such as a point that may be designated 
by a time search. The time search is an operation of locating 
a playback point corresponding to a time input by a user in 
minutes/seconds. The time input is made in a relatively large 

20 unit such as ten minutes and ten seconds. Accordingly, the 
Acquisition Point DS is provided at a point that can be designated 
by a time search made in units of 10 minutes and 10 seconds. 
By providing the Acquisition Point DS at such a point that can 
be designated by a time search, playback of the graphics stream 

25 can be smoothly carried out when a time search is conducted. 

The Normal Case DS provides a display effect "display 
update", and contains only a difference from the immediately 

15 
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preceding display composition. For example, if a DS V is 
identical in contents to an immediately preceding DS U but 
different in display composition, the DS V is a Normal Case DS 
which contains only an ICS or an ODS . This eliminates the need 
to provide overlapping ODS.s in DSs, so that the amount of data 
stored on the BD-ROM 100 is reduced. Since the Normal Case DS 
contains the difference without the overlapping data, no display 
composition can be displayed with the Normal Case DS alone. 

The Epoch Continue DS indicates at the start of an AV clip 
that one Epoch continues from a different AV clip. For example, 
a DS n contains a composition state field having a value indicating 
"Epoch Continue". In this case, the DS n belongs to the same 
Epoch as an immediately preceding DS n -i even if the two DSs are 
on different AV clips. Since the DS n and DS n -i belong to the 
same Epoch, there will be no flushing of the graphics plane and 
of the object buffer even if there is a boundary of AV clips 
between the two DSs. 

In FIG. 5, dashed lines kzl show to which DS the functional 
segments on the fourth level belong. Since the functional 
segments shown in FIGs . 4A and 5 are the same, it is known that 
the functional segments shown in FIG. 4A belong to the Epoch 
Start DS. Functional segments belonging to the Acquisition 
Point DS are identical to those belonging to the Epoch Start 
DS. In addition, functional segments belonging to the Normal 
Case DS are identical to those belonging to the Epoch Start DS, 
except that some of the functional segments are omitted in the 
Normal Case DS . 
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This concludes the description of the logical structure . 
def inedby the functional segments . Next r a description is given 

4 

to the allocation of DSs having such- an ICS and ODSs, to the 
time axis of AV clip playback. Since the Epoch is a time unit 
5 of continuous memory management and includes one or more DSs, 
it is important how those DSs are allocated to the time axis 
of the AV clip playback. Note that the "time axis of the AV 
clip playback" refers to the time axis used for defining the 
decode timing and playback timing of individual pictures 

10 multiplexed in the AV clip . The decode timing andplayback timing 
are expressed in the time accuracy of 90 KHz. The DTS and PTS 
of an ICS or of an ODS within a DS show the timing based on which 
the synchronous control is realized. DSs are allocated to the 
time axis of the AV clip playback by carrying out the synchronous 

15 control in accordance with each DTS and PTS attached to an ICS 
and to an ODS. 

Suppose an arbitrary one of DSs within an Epoch is denoted 
as a DS n , and that the DS n is allocated to the time axis of the 
AV clip playback based on the DTS and PTS set as shown in FIG. 
20 6. 

FIG. 6 shows the time axis of AV clip playback to which 
the DS n is allocated. In the figure, the duration of DS n starts 
at the time shown by the value of DTS of the ICS belonging to 
the DS n (DTS (DS n [ICS] ) ) , and ends at the time shown by the value 
25 of PTS - of the END belonging to the DS n (PTS (DS n [ICS] ) ) . The 
display timing for the first presentation of the DS n is shown 
by the value . of PTS of the ICS (PTS (DS n [ICS] ) ) . By adjusting 
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the PTS (DS n [ICS] ) value to correspond to the display timing of 
a desired picture within the video stream, the first presentation 
of the DS n will be synchronized with the video stream. 

The PTS (DS n [ICS] ) value is the sum of the time taken for 

* 

5 decoding the ODS ( DECODEDURATION) and the time taken for 
transferring the decoded graphics object (TRANSFERDURATION) . 

The decoding of ODS needed for the first presentation is 
completed within the DECODEDURATION. The duration mcl shown 
in FIG. 6 is the time required for decoding an ODS m , an arbitrary 
10 one of ODSs within the DS n . The start of the decode duration 
mcl is shown by the value of DTS (ODS n [ODS m ] ) , and the end of 
the decode duration mcl is shown by the value of PTS (ODS n [ODS m ] ) . 

An Epoch is defined by allocating every ODS within the Epoch 
to the time axis AV playback as described above . This concludes 
15 the description of the allocation to the time axis of AV playback. 

One feature of the first embodiment is to control the 
behavior of a Multi-page menu in accordance with the proceeding 
of AV playback. The novel structure realizing the .feature 
resides in an interactive_composition structure provided in the 
20 ICS. Hereinafter , a description is given to the 

interactive_composition structure . 

FIGs. 7A and 7B are views showing the relations between 
the interactive_composition structures and the ICSs. The 
interactive__composition structure and the ICS may be in 
25 one-to-one correspondence as shown in FIG . 7A, or one-to-multiple 
correspondence -as shown in FIG. 7B. 

The one-to-one correspondence is applicable to the case 
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where the interactive_composition structure is small enough to 
be contained in one ICS. 

4 

The one-to-multiple correspondence is applicable to the 
case where the interact±ve_composition structure is too large 
5 in size to be carried by one ICS and thus fragmented to be stored 
in multiple ICSs. Since the interactive composition structure 
can be fragmented to be stored in multiple ICSs, there is no 
restriction imposed on the size of interactive_composition 
structure. The interactive_composition structure may be as 

10 large as 512 Kbytes, 1 Mbyte, or any number of bytes. As above, 
one interactive_composition structure may correspond to 
multiple ICSs . For the simplicity sake, however, a description 
hereinafter is given on precondition that the ICS and 
interactive_composition structure is in one-to-one 

15 correspondence. 

FIG. 8 is a view showing the internal structure of an ICS. 
The ICS contains one interactive_composition structure or a 
fragment of one interactive__composition structure. As shown 
in the left part of FIG. 8, the ICS is composed of the following 

20 fields: "segment descriptor" showing the segment type "ICS"; 
"video_descriptor" showing the width and height of the video 
plane in pixels and the frame rate of the associated video stream; 
"composition_descriptor" showing the composition__state , which 
is information showing the type of'DS to which the ICS belongs 

25 is whether a Normal Case DS, an Acquisition Point DS, an Epoch 
Start DS, or an Ef f ect__Sequence; and 

"interactive composition data fragment'' showing that whether 
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the ICS is carrying an entire interactive__composition structure 
or a fragment of interactive_composition structure. 

■ 

As indicated by an arrow cul, FIG. 8 shows the internal 
structure of an interactive_composition structure in detail. 
The interactive_composition structure is composed of the 
following fields : ^interactive composition length", 

"stream__model" , "user__interf ace_model", 

"compos it ion_time_out_pts", "selection_time_out_pts", 
"use r _t ime_o u t_du r a t i on "page_inf ormation (1) , (2)... '(i) ... 
(number_of__page--l ) " for each page of the Multi-Page menu 
available for presentation. 

The interactive__composition_length field shows the length 
of the interactive__composition structure. 

The "stream_model" field shows the type of stream model 
of the associated interactive_composition structure. The 
stream model shows how the interactive_composition structure 
is stored on the BD-ROM and how the data is to be processed by 
a buffer (referred to as a "composition buffer") on a playback 
apparatus. Specifically, the stream model shows whether (i) 
the graphics stream is multiplexed with an AV clip and separated 
to be loaded to the composition buffer or (ii) a graphics stream 
is stored as a Sub clip in a separate file from the AV clip and 
preloaded to the composition buffer. The stream_model field 
is provided in the interactive__composition structure because 
the composition buffer needs to perform different processing 
depending on whether the interactive_composition structure is 
preloadedormultiplexed. FIG. 9 is a view showing the processing 
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performed on the "Multiplexed" ICS. In the figure , the video 
stream multiplexed in the AV clip is shown on the fourth level, 
and the graphic stream multiplexed in the AV clip is shown on 
the third level. The graphics stream includes three DSs, which 
are a DSi, a DS 2f and DS 8 . The second level shows the proceeding 
of AV playback. The first level shows contents stored in the 
composition buffer at different times of the AV playback. When 
the current playback point reaches the beginning of DSi 
multiplexed in the AV clip, the DSi is loaded to the composition 
buffer as indicatedby an arrowpf 1 . • As a result, the composition 
buffer stores DSi[ICS] . Then, when the current playback point 
reaches the beginning of DS2 multiplexed in the AV clip, the 
DS2 is loaded to the- composition buffer as indicated by an arrow 
pf2. As a result, the composition buff er stores DS 2 [ICS] . That 
is, in the case where the stream_model is "Multiplexed", there 
are more than one interactive__composition structures. The 
playback apparatus discards a currently stored 
interactive__composition structure and newly loads another 
interactive_composition structure residing at the current 
playback point. Thus, the composition buffer stores one 
interactive_composition structure at a time. 

This discarding and loading operation is especially 
effective at the time of a skip operation. When a skip operation 
is executed, the interactive_composition structure associated 
with the current playback point is no longer necessary. Thus, 
the playback apparatus discards the interactive_composition 
structure currently on the composition buffer . As shown in FIG . 
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9, when a skip operation ami is executed, the composition buffer 
discards the stored contents at the execution timing am2 of the 

* 

skip operation. Then, the DS 8 [ICS] residing on the skip 
destination is loaded to the composition buffer as indicated 
5 by an arrow pf 3 . 

As above, in the case of the "multiplexed" type streammodel, 
the composition buffer needs to discard the stored 
interactive_composition structure as the AV playback proceeds. 
On the contrary, in the case of the "preload" type stream model, 

10 the composition buffer is not required to discard the stored 
interactive_composition structure at all. If the composition 
buffer discards the stored interactive_composition structure 
as the AV playback proceeds, the same interactive__composition 
structure as the discarded one needs to be loaded from the BD-ROM 

15 again. This useless operation of re-loading, the same 
interactive_composition structure results in an extra burden. 
FIG. 10 is a view showing the contents of the composition buffer. 
FIG. 10 is similar to FIG. 9 with respect to what is shown in 
each of the first to fourth level. Yet, FIG. 10 is totally 

20 different from FIG. 9 with respect to the contents of the 
composition buffer stored at different times. Specifically, 
even if the current playback point reaches where the DSi is 
multiplexed, the ICS of the DS X is not loaded to the composition 
buffer (indicated by the mark "X" in the figure) . 

25 Similarly, even if the current playback point reaches where 

the DS 2 is multiplexed, the ICS of the DS 2 is not loaded to the 
composition buffer (indicated by the mark "X" in the figure) . 
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Similarly, the contents stored in the composition buffer are 
retained rather than discarded, even if a skip operation is 

4 

executed. The interactive_composition structure is provided 
with the stream_model field for allowing a controlling subject 
5 (the graphics controller) to accurately judge whether to discard 
the stored interactive_composition structure as the proceeding 
of AV playback or to retain the stored interact ive_compos it ion 
structure irrespective of the proceeding of AV playback. This 
concludes the description of the stream_model field. 

10 Referring back to FIG . 8, the "user_interf ace_model" shows 

the type of user interface ' model to be used for the 
interactive_composition structure. The user_interface__model 
shows either "Always-On U/I" or "Pop-Up U/I" . When the Always-On 
U/I is used, a menu appears on screen and disappears in accordance 

15 with the proceeding of AV clip playback. When the -Pop-Up U/I 
is used, a menu appears on screen and disappears in response 
to a user operation. 

The composition__time_out_pts field shows the end time of 
an Epoch (Epoch END) to which the ICS belongs. The interactive 

20 control defined by the ICS is no longer valid at the Epoch END. 
That is, a point of time shown by the composition_time_out__pts 
field is the end of valid interactive control period. 

The " select ion_time__out_pts" field shows the button 
selection termination time. At the time shown by the 

25 select ion_time__out_j?ts value, the button that is currently ±n 3 
the selected is automatically activated. Note that buttons are 
elements representing options in the multi-page menu. The 
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selection_time_outj?ts field defines the valid interaction 
period for user interaction to activate the buttons. 

* 

The IF statement (if ( streainjrriodel == ^0*/ ) ) shown in the 
figure means that the above described 

5 " compos it ion_time_out_pts " and "s elect ion_time_out_pts" 
fields are optional information that are provided only when the 
"stream_model" field shows "Multiplexed" type. In the case of 
ICS of which stream model is "Preloaded" type, neither of the 
"compos it ion_time__out_pts" and "selection_time_out_pts" 
10 fields is provided. 

The user__time__out_duration field shows the timeout at which 
on-s ( creen pages presented in response to a user operation are 
removed. When the Always~On U/I is used, pages from the second 
page onward (referred to as sub pages) are. presented in response 
15 to user operations. Thus, after the timeout defined by the 

* 

use r_t ime_out_duration , all subpages are removed from the screen 
leaving the first page on-screen. When the Pop-Up I/O is used, 
not only the sub pages but also every page of the Multi-page 
menu is presented in response to user operations. Thus, after 
20 the user_time_out_duration, all on-screen pages are removed 
leaving no page remain presented (No Menu Display) . 

Next, a description is given to the meaning of the 
selection_time_out_pts and composition_time_outjpts fields in 
an Epoch. 

25 FIG. 11 is a timing chart illustrating* the functions of 

the selection_time_out_pts andcomposition_time_out__pts fields 
in an ICS belonging to a DS n in the case where the DS n is an 
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Epoch Start DS (n = 1) . The timing chart shows that the 
composition_time__out_pts value specifies the end of Epoch (Epoch 
END) . The interactive display is valid for a period from the 
PTS (DS n [ICS] ) to the Epoch END. This period is referred to as 
the "Interactive Display Period". 

The selection time out_jpts specifies a time preceding the 
Epoch End. U'ser interaction may be made during a period from 
the PTS ( DS n [ ICS] ) to the time specified by the 
selection_time_out_pts value. This period is referred to as 
the "valid interaction period". As above, the 

selection_time_out_pts field defines the end time of a period 
which starts from the first presentation of the interactive 
display and during which the interactive display is available 
for user interaction. The composition__time_out_pts field 
defines a period starting from the first presentation of the 
interactive display during which the interactive display in an 
Epoch remains valid and thus presented. This concludes the 
description of the selection_time_out_pts and 

composition_time_out_pts fields. Next, a description is given 
to the Multi-page menu state transition defined by the 
select ion_time_out_pts, composition_time_out_pts, and 
user_time_out_duration fields . 

FIG. 12A is a view showing the state transition of the 
Multi-page menu based on the s elect ion_time_out_j?ts , 
composition__time_out_pts , ^and user_time_out_duration fields. 
FIG. 12A is a view showing the state transition of the Multi-Page 
menu when the Pop-Up U/I is used. The Multi-page menu shown 

■ 
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in the figure has the following three states: No Menu Display,, 
1 st Page Only, Multi-Page Exist. 

The 1 st page only is the state where only the first page 
of the Multi-Page menu is presented. 
5 The Mult i- Page exist is the state where pages from the second 

page onward (i.e. sub pages) are presented in addition to the 
first page. 

An arrow jtl represents the transition from the No Menu 
Display state to the 1 st Page Only state. This state transition 

10 is triggered by an event w Pop-Up_On" . The "Pop*-Up_On" is an 
event generated within the playback apparatus in response to 
a user operation for a menu call. 

An arrow jt2 represents the transition from the 1 st Page 
Only state to the No Menu Display state . This state transition 

15 is triggered by an- event "Pop-Up_Of f " . The "Pop-Up_Of f " is an 
event generated within the playback apparatus in response to 
a user operation for a menu removal. 

Arrows jt3 and jt4 represent the transition from the 1 st 
Page Only state to the Multi-Page Exist state, and then from 

20 the Multi-Page Exist state to the 1 st Page Only state. The state 
transition is triggered by an event "ChangeButtonPage" generated 
upon the timeout defined by the selection_time_out_pts . The 
ChangeButtonPage is an event generated within the playback 
apparatus in response to a user operation for switching a menu 

25 page. The event is generated on precondition that a user 
operation for switching a menu page is made. The 
s elect ion_time_out_jpts shows the time for forcefully 
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invalidating a button for receiving a user operation for 
switching a menu page. Thus, the description of the 
selection_time_out_pts field realizes the state transition of 
"1 st Page Only -> Multi Page Exist - 1 st Page Only". 
5 An arrow jt5 represents the transition from the 1 st Page 

Only state to the No Menu Display state . An arrow j 1 6 represents 
the transition from the Multi-Page Exist state to the No Menu 
Display state. The state transition jt6 is triggered by the 
timeout of the user time out duration. 

10 In the figure, the state transitions to the No Menu Display 

state from the 1 st Page Only state ( jt5) and from the Multi-Page 
Exist state are defined by the userjtime_out_duration field. 
That is to say, it is possible to define such control that on-screen 
menu pages are automatically removed when the AV playback reaches 

15 to a climax scene. With this control, it is avoided that the 
menu pages interferes with viewing the video stream playback. 

FIG. 12B shows the state transition of Multi-page menu when 
the Always-On U/I is-used . An arrow at 1 represents the transition 
from the No Menu Display state to the 1 st Page Only state. This 

20 state transition is triggered by an event "Epoch Start". The 
"Epoch Start" is an event showing that the current playback point 
reaches the point corresponding to the PTS associated with the 
interactive_composit ion structure . It is shown that the 1 st page 
is automatically presented as the video stream playback proceeds 

25 and without waiting for any user operation. 

An arrow at2 shows the transition from the 1 st Page Only 
state to the Multi-Page Exist state, and an arrow at3 shows the 
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transition from the Multi-Page Exist state to the 1 st Page Only 
state. These state transitions are also triggered by the 
ChangeButtonPage event generated upon a timeout of the valid 
interaction period defined by the selection_time__out_pts . 
5 An arrow at 4 represents the transition from the Multi-Page 

Exist state to .the 1 st Page Only state. This state transition 
is triggered by the timeout of the user_time_out_duration . As 
apparent also from this state transition, when the Always-On 
U/I is used, the state transition is made to the 1 st Page Only 

10 state not to the No Menu Display state upon the time out of the 
user_t ime_out_durat ion . 

An arrow at5 represents the transition from the 1 st Page 
Only state to the No Menu Display state. This state transition 
is triggered by the time out of the composition_time_out_pts . 

15 Note that the composition_time_out_pts shows the end of Epoch 
(EpochEND) . 

As apparent from FIGs. 12A and 12B, the Multi-Page menu 
behavior is defined by the selection_time_out_pts , 
user_time_out_duration, and composit±on_t±me_out__pts fields 

20 provided in the interactive_composition structure. 

FIGs . 13-15 show the screen display presented under control 
defined as above. FIG. 13 shows the display scenario 
incorporating the Multi-Page menu. In this scenario, the 
Multi-Page menu is not presented on screen at first. The 1 st 

25 page of the Multi-Page menu is presented when the current playback 
point reaches the point tl, and a Sub page is additionally 
presented at the point t2. When the current playback point 
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reaches the point t3, the on-screen Sub page is removed leaving 
only the 1 st page on screen. The 1 st page is then removed at 

« 

the point t4' leaving no menu page displayed on screen. 

At the bottom of FIG. 13 is the interactive_composition 

r 

structure described to achieve the above display scenario. The 
PTS (DS n [ICS] ) is set to the value corresponding to the point 
tl, the select ion_time_out_pts to the value corresponding to 
the point t2, the user_time_out_duration to the value 
corresponding to the point t3, and the composition_time_out_pts 
to the value corresponding to the point t4. With this setting, 
the Multi-Page menu undergoes the state transitions shown in 
FIG. 13. 

Now, a description is given to the setting of 
selection_time__out_pts value in greater detail with reference 
to a specific example. 

FIGs. 14A-14C show a display example defined by the 
selection_time__out_pts . As shown in FIG. 14A, at the point of 
tl, the first page of Multi-Page Menu is overlaid with a picture 
of the video stream. A picture presented at the point t2 is 
of a scenic shot and is of less importance in the story of the 
AV clip. Since the picture presents no valuable contents for 
users, the selection_time_out_pts value is set to automatically 
activate a button presented on the 1 st Page at the point t2 . 
Consequently, a Sub Page is overlaid with the scenic short shown 
in FIG. 14B, so that a composite image as shown in FIG. 14C appears 
on screen. 

In a similar manner, a description is given to the setting 
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of user_time_out_duration value with reference to a specific 
display example. FIGs . 15A-15D show a display example defined 
by the user_time_out_duration . In this example, the Multi-Page 
menu is in the Multi-Page Exist state- where multiple Sub Pages 
(2 nd , 3 rd , and 4 th pages) are presented on screen as shown in FIG. 
15A. In the Multi-Page Exist state, the pages of the Multi-Page 
menu are overlaid with a picture included in the video stream. 
Here, the picture to be presented at the point t3 is a character 
image as shown in FIG. 15B. In this case, the multiple Sub Pages 
overlaid with the picture end up covering most of the character 
image as shown In FIG. 15C. As a result, the facial expression 
of the character image can not be seen. To avoid this 
interference, the user_tirue_out__duration is set to the value 
corresponding to a time immediately before the point t3 . With 
this setting, the Multi-Page menu undergoes the state transition 
from the Multi-Page Exist state to the 1 st Page Only state. As 
a result, the composite image as shown in FIG. 14D is presented 
at the point t3 . Here, the 1 st page is solely presented on screen 
without blocking the character image too much. Furthermore, 
since the 1 st page remains on screen rather than removing every 
page of the Multi-Page Menu, there is no need to call the menu 
again . 

As described above, according to the first embodiment, the 
ICS multiplexed in the graphics stream includes control 
information defining behavior of a Multi-Pdge Menu in accordance 
with the proceeding of AV playback. This makes it possible to 
define such control that numerous pages are presented during 
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the playback of less important scenes, such as scenic shorts, 
and that on-screen pages are removedas the AV playback approaches 
climax scenes. With such control, the-menu pages are presented 
without interfering with the playback of the AV clip, which is 
5 the main content. Thus, the user can concentrate on viewing 
the AV playback while enjoying convenience of the Multi-Page 
Menu functions. 

(SECOND EMBODIMENT) 

10 A second embodiment relates to display compositions for 

presenting pages of a Multi-Page Menu. According to the second 
embodiment, t a display composition is designed to present a 
plurality of graphical elements called button on screen. In 
addition, an animation effect is presented on screen before 

15 introducing and/or after removing a menu page. 

FIG. 16 is a view showing the internal structure of page 
information associated with an arbitrary one of multiple pages 
(x th page) of the Multi-Page Menu. As shown in the left part 
of the figure, page information (x) is composed of the following 

20 fields: u page_id" showing an identifier uniquely identifying 
the page(x); "UO_mask_table"; "in_ef f ects" specifying an 
animation effect to be presented as introduction of the page 
x to the screen; "out_ef f ects" specifying an animation effect 
to be presented after removal of the page(x) from the screen; 

25 n animation_frame_rate__eode // showing an animation frame rate to 
be followed at the time of presenting animation effects for the 
page(x); xx default selected button id ref", 



31 



WO 2005/048261 



PCT/JP2004/017215 



"def ault__activated_button_id_ref " ; "pallet_id_ref " ; and 
"button_info (1), (2)... (number_of_buttons-l ) " each associated 
with one of the buttons presented on the page(x). 

The UO_mask_table field shows permissions and prohibitions 
of user operations on the page (x) . If this field is set to a 
value showing "prohibited", a corresponding user operation to 
the playback apparatus is invalid while the page(x) is active. 

The def ault_s elect ed_button_id__ref field shows whether a 
button to be selected as default upon starting the presentation 
of page(x) is determined dynamically or statically. When this 
field is set to "OxFF", the default selected button is determined 
dynamically. In such a case, the value held in a Player Status 
Register (PSR) of the playback apparatus is interpreted with 
priority, and the button shown by the PSR is put into the selected 
state. On the other hand, when this field is set to the value 
other than "OxFF" , the default selected button is determined 
statically. In such a case, the PRS is overwritten with the 
def ault__selected_button_id_ref field value, and the button 
specified by this field is put into the selected state. 

The def ault_activated__button_id_ref field shows a button 
to be automatically activated at the time defined by the 
selection__time__out_pts field value. When the 

def ault_activated_button__id_ref field value is set to "FF", the 
button that is in the selected state at the time of the timeout 
is automatically activated. On the other hand, when this field 
is set to "00", no button is automatically activated. When this 
field is set to the value that is neither "00" or "FF", the value 
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is interpreted as a valid button number statically specifying 
the button to be automatically activated. 

The pallet_id__ref field shows the ID associated with a 
pallet to be set in a CLUT unit, which will be described later. 
5 Each butt on_inf o structure is information defining a button 

to be presented on the page(x) . With the above fields, each 
page of the Multi-Page menu is defined. Next, a description 
is given to the internal structure of button_info structure. 
In the description, an arbitrary one of the buttons on the page (x) 
10 is denoted as a button (i) . In FIG. 16, arrows cxl indicate that 
the internal structure of button_inf o (i) is excerpted to be shown 
in detail. 

Each button presented on the page has three states which 
are a normal state, a selected state, and an activated state. 

15 When a button is inthenormal state, thebuttonis simply displayed. 
When a button is in the selected stated, the button is currently 
focused as a result of a user operation but not yet activated. 
When a button is in the activated state, the button has been 
activated. Since each button has these three states, the 

20 button_inf o (i) is provided with the following information. 

The button_id field shows a value uniquely identifying the 
button (i) in the interactive_composition structure. 

The button_numeric_select_value field shows a flag 
indicating whether the button (i) is numerically selectable. 

25 The auto_action_f lag field shows whether the button (i) is 

automatically activated. When the auto_action_f lag field is 
set to ON (bit value of "1") , the button (i) is transferred not 
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to the selected state but directly to the activated state when 
selected. On the other hand, when the auto_action_f lag field 
is set to OFF (bit value of "0") , the button (i) is transferred 
not to the activated state but to the selected state when selected . 

The button_horizontal__position - field and 

button_vertical_position field respectively specify the 
horizontal and vertical positions of the top left pixel of the 
button (i) on the interactive display. 

The neighbor_inf o structure shows buttons to receive the 

, .. 

selected state in response to user operations made to move a 
focus in an up, down, right, and left direction when the button (i) 
is in the selected state. The neighbor_inf o structure is 
composed of upper_button_id_ref , lower_button_id_ref , 
lef t_button__id_ref , and right_button_id_ref fields. 

The upper_button__id_ref field specifies the button number 
of a button to receive the selected state if a user operation 
instructing to move the focus in the up direction is made while 
the button (i) is in the selected state. The move up operation 
is executed at a push of a Move Up key of the remote controller. 
If this field is set to the same button number as that of the 
button (i), the user operation to the Move Up key is ignored. 

Similarly, the lower_button_id_ref field, the 
lef t_button_id_ref field, the right_button__id_ref field each 
specify the button numbers of a button to receive the selected 
state if a user operation instructing to move the focus to the 
down, left, or right direction is made, while the button (i) is 
in the selected state. The respective operations are executed 
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at a push of the Move Down key r Move Left key, and Move Right 
key of the remote controller. If these fields are set to the 
same button number as that of the button (i) r the user operations 
to the those keys are ignored. 

The normal_state_info structure defines the normal state 
of the button (i), and is composed of the 
normal__start_obj ect_id_ref f normal_end__ob j ect__id_ref , and 
norma l_repeat_f lag fields . 

The normal__start_obj ect__id_ref field specifies the first 
one of the object__id values that are serially assigned to a 
sequence of ODSs used to present the normal state of button (i) 
in animation. 

The normal_end__ob j ect__id_ref field specifies the last one 
of the obj ect_id values that are serially assigned to the sequence 
of ODSs used to present the normal state of button ( i ) in animation . 
If the norma l_end_obj ect_id_ref field specifies the same ID value 
as that of the normal_start_ob j ect__id_ref , the static image of 
a graphics object identified by that ID is presented as the 
button (i) . 

The normal_repeat_f lag field specifies whether the 
animation of the button (i) in the normal state is to be 
continuously repeated. 

The selected__state__inf o structure defines the selected 
state of the button (i) . The selected__state__inf o structure is 
composed of the selected__state_sound__id_ref r 

selected__start_ob j ect_id__ref , selected_end_ob j ect_id_ref , 
and selected_repeat__f lag fields. 
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The selected_state_sound_id_ref field specifies sound 
data to be reproduced as a click sound when the button (i) is 
put to the selected state. The sound data is specified by the 
sound_id of a piece of sound data contained in the file called 
5 sound. bdmv. When this field is set to "OxFF", no sound data 
is associated with the selected state of the button (i) and thus 
no click sound is reproduced when the button (i) is changed to 
the selected state. 

The selected_start_ob j ect_id_ref field specifies the 

10 first one of the object_id values that are serially assigned 
to a sequence of ODSs used to present the selected state of 
button (i) in animation. 

The selcted_end_ob j ect_id__ref field specifies the last one 
of the object_id values that are serially assigned to the sequence 

15 of ODSs used to present the normal state of button (i) in animation . 
If the selected__end_ob j ect__id__ref field specifies the same ID 
value as that of the selected_start_ob j ect_id_ref , the static 
image of a graphics object identified by the ID is presented 
as the button (i) . 

20 The selected_repeat_f lag field specifies whether the 

animation of the button (i) in the selected state is to be 
continuously repeated. If the selcted_start_ob j ect__id__ref and 
selcted__end_ob j ect__id_ref fields have the same value , the 
selected_repeat_f lag field is set to the value of x> 00". 

25 The activated_state_inf o structure defines the activated 

state of the button (i) , and is composed of the 
activated state sound id ref, activated start object id ref, 



36 



WO 2005/048261 



PCT/JP2004/017215 



and activated_end___obj ect_id__ref fields . 

The activated_state_sound_id__ref field specifies sound 
data to be reproduced as a click sound when the button (i) is 
put to the activated state. The sound data is specified by the 
5 sound_id of a piece of sound data contained in the sound, bdmv 
file. When this field is set to "OxFF" , no sound data is 
associated with the activated state of the button (i) and thus 
no click sound is reproduced when the button (i) is changed to 
the activated state. 

k 

10 The activated_start__ob j ect_id_ref field specifies the 

first one of the object__id values that are serially assigned 
to a sequence of ODSs used to present the activated state of 
button ( i ) in animation . 

The activated end_ob j ect_id_ref field specifies the last 

15 one of the object_id values that are serially assigned to the 
sequence of ODSs used to present the activated state of button (i) 
in animation. 

The "navigation__command" structure shows a navigation 
command to be executed when the button (i) is activated. A 

20 representative example of the navigation command is a 
SetButtonPage command . The SetButtonPage command instructs the 
playback apparatus to display a desired page of the Multi-Page 
menu with a desired one of buttons on the page in the selected 
state. With the use of such navigation commands, content 

25 creators can readily* describe page transitions at the time of 
authoring. 

This concludes the description of the button_inf o structure . 
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Now, a reference is made to FIG. 18 showing an example of the 
button_inf o description defining the state transition of buttons 

4 

O-A to O-D, as illustrated in FIG. 17. In FIG. 17, arrows hhl 
and hh2 represent the state transitions defined by the 
5 neighbor_inf o () structure of the button info(l), which is 
associated with the button O-A. In the button info(l), the 
lower_button_id_ref "field is set to the value specifying the 
button O-C. Thus, if a user operation is made to the Move Down 
key (FIG. 17, upl) while the button O-A is in the selected state, 

10 the button O-C receives the selected state (sjl). The 
right_button_id_ref field in the button info(l) is set to the 
value specifying the button O-B. Thus, if a user operation is 
made to the Move Right key (up2) while the button O-A is in the 
selected state, the button O-B receives the selected state (sj2) . 

15 An arrow hh3 in FIG. 17 represents the button state 

transition defined by the neighbor_inf o ( ) structure in the 
button_inf o (2 ) , which is associated with the button O-C. In 
the button_inf o (2 ) , the upper_button_id_ref field is set to the 
value specifying the button O-A. Thus, if a user operation is 

20 made to the Move Up key (up3) while the button O-C is in the 
selected state, the button O-A is put back into the selected 
state (sj3) . 

Now, a description is given to the graphical images of the 
buttons O-A, O-B, O-C, and O-D. A DS n provided with the ICS 
25 shown in FIG. 18 includes ODSs 11-19 corresponding to the 
graphical images shown in FIG. 19. The 

normal start object id_ref and normal end object id ref 
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fields in the button_inf o ( 0 ) specify the ODSs 11 and 13,, 
respectively. Thus, the normal state of the button O-A is 
presented in animation using the sequence of the ODSs 11-13. 
Similarly, the selected_start_ob j ect__id__ref and 

5 selected_end__obj ect_id_ref fields in the button_inf o ( 0 ) 
specify the ODSs 14 and 16, respectively. Thus, the selected 
state of the button O-A is presented in animation using the 
sequence of the ODSs 14-16. With this se-tting, when a user 
operation is made to put the button O-A to the selected state, 

10 the graphical image serving as the button O-A is changed from 
the one presented using the ODSs 11-13 to the one presented using 
the ODSs 14-16. Here, if the normal_repeat_f lag field in the 
normal_state_inf o ( ) structure is set to the value "1", the 
animated presentation of the ODSs 11-13 is repeated as indicated 

15 by "-(A)" and (A) " in the figure. Similarly, if the 
selected_repeat__f lag field in the selected_state_inf o ( ) 
structure is set to the value "1", the animated presentation 
of the ODSs 14-16 is repeated as indicated by M ->(B)" and "->(B) " 
in the figure. 

20 As above, a plurality of sets of ODSs that can be presented 

in animation are associated with the buttons O-A, O-B, 0-C, and 
O-D. Thus, with the ICS referencing the ODSs, such a control 
is defined that a character image serving as a button changes 
its facial expression in response to user operations. 

25 This concludes the description of the button__inf o structure . 

Next, a description is given to animation effects. As shown 
in FIG . 16, the in-effects field and the out-effects field each 
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specify an ef f ect__sequence structure . FIG. 20 is a view showing 
an effect_sequence structure. As shown in the left half of the 
figure, the ef f ect_sequence structure is composed of: 
window_inf o ( 0 ) , (1), (2)... (nuraber_of_windows-l) ; and the 
5 ef f ect_inf o ( 0 ) , (1), (2)... (number_of_ef f ects-1) . 

The animation effect is presented by updating display 
compositions at fixed intervals. Each effect_info structure 
is a piece of information defining one display composition. 
Arrows eel indicate that the internal structure of effect__info 

10 is excerpted to be shown in detail. As shown in the figure, 
the effect_info structure is composed of: the ef f ect_duration 
field showing the above-mentioned fixed interval, i.e. a time 
period for which the associated display composition is to be 
presented; the pallet_id_ref field specifying a pallet used for 

15 the associated display composition; and the composition_ob j ect 
(0) , (1) , (2) ... (number_of__composition_obj ect-1 ) specif ying the 
details of the display composition. 

Each window_info structure defines a window or an area on 
the graphics plane within which the display compositions .are 

20 rendered. As indicated by arrows wcl, the window_info is 
excerpted to be shown in detail. As shown in the figure, the 
window__info is composed of the following fields: ^window^id" 
uniquely identifying a widow on the graphics plane; 
"window_horizontal_position" specifying the horizontal 

25 position of the top left pixel of the window; 
^window^vertical^position" specifying the vertical position of 
the top left pixel of the window; "window_width" specifying the 
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width of the window; and xx window_height" specifying the height 
of the window. 

Now, a description is given to values that may be set in 
each of the window_horizontal_position, 

window_vertical_position, window_width, and window__height 
fields. These fields are set to the value corresponding to the 
coordinates within the two-dimensional graphics plane having 
the height and width defined by the video_height and video_width 
fields . 

Since the window_horizontal_position field shows the 
horizontal address of the pixel on the graphics plane, this field 
takes on a value from "1" to the video_width value. Similarly, 
since the window__vertical_position shows the vertical address 
of the pixel on the graphics plane, this field takes on a value 
from "1" to the video__height value. 

Since the window_width field shows the width of a window 
on the graphics plane, this field takes on a value from "1" to 
the value calculated by subtracting the 

window_horizontal_position value from the video_width value. 
Similarly, since the window_height field shows the height of 
window on the graphics plane, this field takes on a value from 
"1" to the value calculated by subtracting the 
window_vertical_jposition value from the video height value. 

As above, the window_info is provided with 
window_horizontal_position, window_vertical_pbsition, 
window__width, window_height fields defining the size andpostion 
of a window on the graphics plane. 
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Now, a description is given to the constraint on the window 
size for realizing the framework for presenting display 
compositions of graphics at a rate of 24'frames per second. This 
frame rate corresponds to the video frame rate applied to the 
5 video stream playback. With this framework, the graphics 
ob j ects are presented in detailed synchronization with the video 
stream. To realize the framework, the window must be cleared 
and rendered at the video frame rate. Here, a consideration 
is given to the required transfer rate from the object buffer 
10 to the graphics plane. 

First, a constraint on the window size is examined. Let 
Rc denote the transfer rate from the obj ect buffer to the graphics 
plane. In the worst-case scenario, the clearing and rendering 
of the window need to be performed at Rc. In other words, the 
15 clearing and rendering need to be performed at half of-Rc (Rc/2) . 

Thus, the following equation needs to be satisfied. 

(Window Size) * (Frame Rate) =? Rc/2 

20 When the frame rate is 24, then RC is expressed in the 

following equation . 

Rc = (Window Size) x 2 * 24 

25 When the total number of pixels of the graphics plane is 

1920 x 1080 and the bit length of an index per pixel is 8 bits, 
the total capacity of the graphics plane is 2 Mbytes (=? 1920 
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x 1080 x 8) . 

When Rc is 128 Mbps and the window size is 1/A of the graphics 
plane, the following equation is satisfied. 

5 128,000,000 = 1920 * 1080 x 1/A x 8 x 2 x 24 

1/A = 0.16 (= 128,000,000/(1920 x 1080 x 8 x 2 x 24)) 

* 

Thus, by restricting the window size to 16% of the entire 
graphics plane, the synchronization between animation effects 

10 with the video stream is guaranteed as long as the animation 
effects are rendered at the transfer rate of 128 Mbps. 

Suppose, for example, the ef f ect__duration showing the rate 
of clearing and rendering a window is 1/2 or 1/4 of the video 
frame rate. In this case, the window size can be as large as 

15 32% or 64% of the entire graphics plane even if Rc is the same. 
That is to say, the effect_sequence structure has a feature that 
the ef f ect_duration is determined to be longer for a larger window 
size, and shorter for a smaller window size. This concludes 
the description of the window size . Next, a description is given 

20 to the position of window. As mentioned above, the position 
and the size of a window are fixed throughout an Epoch. 

The position and the size of a window are fixed throughout 
the duration of an Epoch for the following reason. When the 
position or size of a window is changed, the write address of 

25 the graphics plan£ needs to.be changed, which incurs overhead. 
The overhead inevitably reduces the transfer rate from the ob j ect 
buffer to the graphics plane. 
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There is a constraint on the number of graphics objects 
allowed to be rendered in one window at a time. The number of 
graphics objects is restricted in order to reduce overhead 
incurred at the time of transferring decoded graphics objects. 
5 To be more specific, the overhead referred herein is incurred 
at the time of setting addresses of edge parts of the graphics 
objects. This overhead increases if the number of edge parts 
is greater. 

If there is no restriction on the number of graphics ob j ect s 
10 that can be displayed in one window, the overhead is incurred 
unlimitedly when transferring graphics objects to the graphics 
plane, which increases fluctuations in transfer load. On the 
other hand, if the number of graphics objects in one window is 
limited to two, the transfer rate Rc can be set on an assumption 
15 that the number of instances of overhead is at most four. Hence 
' a minimum standard for the transfer rate Rc can be determined 
easily. 

Turning now to the effect_info structure, the 
ef f ect_duration is determined depending on the window size . Now, 

20 a description is given to how the ef f ect_duration is determined. 
FIG. 21A is a view schematically illustrating how the 
ef f ect_duration is determined. In the figure, the (j + l) th 
display composition is presented after a plurality of video 
frames from the presentation of the j th display composition. 

25 The number of the plurality of video frames is the ef f ect_duration. 
In order to calculate the ef f ect_duration, the total size of 
data to be rendered for presentation of each display composition 
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must be known. Here, the effect_sequence is provided with 
windows of the number that is indicated by the number__of ^windows 
field value. Thus, the total size of data needed to be rendered 
for presentation of the j th display composition is equal to the 
5 total size of the windows (0) through (number_of_windows-l) , 
as indicated by the brace br. 

As above, the j th display composition is provided with the 
windows (0) through (number_of__windows-l ) . Thus, the total 
data size of the windows is divided by the transfer rate of the 

10 playback apparatus (128 Mbps) and further multiplied by the time 
accuracy of PTS (90 KHz), so that the result is expressed in 
90 KHz. The resulting value is the duration for which the j th 
display composition of the effect_sequence is presented. FIG. 
21B shows the equation for calculating the ef f ect_duration . 

15 Note that in this specification, the operator "ceil()" 

represents a divide that returns a value rounded up to an 
integer . 

FIG. 22 is a view showing a specific example of an in-effect 
animation. The in-effect animation presents an image such that 

20 a flash of light appears on the palm of hand of the character 
image on screen and a menu gradually rises up after the flash 
of light disappears. The size of graphics data is the largest 
within the in-effect at the time when the menu is fully shown 
on screen (effect (4)). Thus, the window_width and 

25 window_height of a window are determined based on the data size 
of the effect (4) . Suppose, the window size determined herein 
is about 50% of the entire graphics plane. This window size 
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is nearly three times as large as the above-mentioned window 
size of 16% of the graphics plane. Consequently, the 
ef f ect_duration for this in-ef feet is- determined to be longer 
in order to allow the display composition to be duly updated 
5 at the rate of 12 8 Mbps . Each ef f ect_duration of the effects 
(0) , (1) , (2) , and (3) is set to be relatively long. According 
to this setting, each display composition is rendered for every 
three video frames, so that it is ensured that the in-ef feet 
animation is seamlessly presented. The same scheme applies to 

10 an out-effect animation . FIG. 23 is a view showing the out-effect 
animation for which window_width, window_height , and 
ef f ect_duration field values are determined in the manner 
described above. 

As described above, according to the second embodiment, 

15 the window_info defines an area in the graphics .plane for 
rendering, and the eff ect_info defines a duration at intervals 
of which the rendering area is updated. Thus, the duration may 
be adjusted to be longer when the rendering area is set to be 
larger, and shorter when the rendering area is set to be smaller. 

20 With this setting, the seamless presentation of animation effects 
is ensured as long as the playback apparatus transfers graphics 
data at a fixed transfer rate. 

(THIRD EMBODIMENT) 

4 

25 A third embodiment of the present invention relates to an 

improvement on the presentation of animation effects, such as 
Scroll, Wipe-In/Out, and Cut-In/Out. The structure for 
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achieving the animation effects lies in the compos ition_ob j ect 
structures shown in FIG. 20. FIG. 24 is a view showing the 

4 

internal structure of an arbitrary composition_ob j ect ( i ) in 
detail. As shown in the figure, the composition_ob j ect (i) is 
composed of the following fields: "obj ect__id__ref " ; 
xv window_id_ref ", ^ "obj ect_cropped__f lag"; 

" compos it ion_obj ect_horizontal_j?osition" ; 

"composition_ob j ect_yertical_position" , and 
"cropping_rectangle__inf o ( 1 ) , (2)... (n) " . 

The ob j ect__id_jref field shows a reference value for a 
graphics object identifier (object_id) . The reference value 
corresponds to the object_id identifying the graphics object 
that is used for generating a display composition according to 
the composition_ob j ect (i) . 

The window__id_ref field shows a reference value for a window 
identifier (window__id) . The reference value corresponds to the 
window_id identifying the window in which the display composition 
according to the composition_ob j ect (i) is rendered. 

The ob j ect__cropped_f lag field shows a flag indicating 
whether the graphics object cropped on the object buffer is to 
be presented- or not to be presented. When this field is set 
to the graphics object cropped on the object buffer is 

presented. On the other hand, when this field is set to "0", 
the cropped graphics object is not presented. 

The composition_ob j ect__horizontal_position field shows 
the horizontal postion of the top left pixel of the graphics 
object on the graphics plane. 
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The composition_object_vertical_position field shows the 
vertical position of the top left pixel of the graphics object 
on the graphics plane. 

The cropping_rectangle structure is valid when the 
5 object_cropped_f lag field is set to "1" . In FIG. 24, arrows 
wd2 indicate that the internal structure of cropping rectangle 

* - 

is excerpted to be shown in detail. As shown in the figure, 
the cropping_rectangle structure is composed of the following 
fields : "obj ect_cropping_horizontal_position" ; 

10 M obj ect_cropping_vertical_jposition" ; 

r 

xx object_cropping_width"; and "obj ect_cropping_height" . 

The obj ect_cropping_horizontal_position field shows the 
horizontal position of the top left corner of the cropping 
rectangle. The cropping rectangle defines the region of the 
15 graphics object to be cropped. . 

The object_cropping_vertical_postion field shows the 
vertical postion of the top left corner of the cropping rectangle . 

The object_cropping_width field shows the width of the 

■ 

cropping rectangle. 

20 The obj ect_cropping_height field shows the height of the 

cropping rectangle . 

This concludes the description of the structure of 
composition_obj ect . Next, a description is given to a specific 
example of the compos it ion_ob j ect structure. 

25 FIG. 25 is a view showing a specific example of an in-effect 

presenting a Right-Scroll animation effect. The in-effect 
animation of this example presents an image such that a language 
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selection menu gradually appears from the right edge of the screen . 
For this in-effect animation, compos ition_ob j ect ( 0 ) , 

i 

compos.it ion_obj ect (1) , compos ition_ob j ect (2) , and 

composition_object ( 3 ) are allocated to points tO, tl, t2, and 
t3, respectively- Furthermore, the ICS and the effect_info 
structure associated with each of the DSs are defined to include 
the following window_info structures and composition object 
structures. 

Now, a description is given to the setting of each 
composition__obj ect structure. FIGs. 26-29 each illustrate an 
example setting of the composition__ob j ect structure. FIG. 2 6 
is a view showing an example of the composition__ob j ect ( 0 ) 
structure . 

In the figure, the window_horizontal_position and 
window_verticaljposition fields specify the coordinates of the 
top left pixel LPO of a window on the graphics plane. The 
window_width and window_height fields specify the width and 
height of the window. 

As shown in the figure, the 

obj ect_cropping__horizontal_position and 
object_cropping_vertical_position fields specify a reference 
point ST1 used for determining a region of the graphics object 
to be cropped. The reference point ST1 is addressed based on 
the coordinate system which has an origin at the top left corner 
of the graphics^ object on the object buffer. The region having 
the length and width specified by the ob j ect__cropping__width and 
object__cropping_height fields from the reference point ST1 is 
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determined as the cropping rectangle (solid-line box in the 
figure) . The cropped part of the graphics object is rendered 
on the graphics plane at the position having the top left pixel 
LPO specified by the composition^ob j ect_horizontal_position 
5 and composition_object_vertical_position fields. In this 
example, about 1/4 of the language selection menu from the right 
is rendered within the window on the graphics plane. As a result, 
the right one-quarter of the language selection menu is presented 
as a composite image overlaid on the video. 

10 FIGs . 27, 28, and 29 are views showing examples of the 

composition__object structure (1) , composition_ob j ect 

structure (2) , and composition_ob j ect structure (3) , 

respectively. The window_info structure shown in each of the 
figures is identical with the one shown in FIG . 26. Thus, the 

15 description is omitted. The composition^ object (1). structure 
shown in FIG. 21, however, differs from the one shown in FIG. 
26. In FIG. 27, the ob j ect_cropping_horizontal_position and 
ob j ect_cropping_verticalj?osition fields specify the 
coordinates of the top left pixel of about a right half, rather 

20 than 1/4, of the language selection menu stored on the object 
buffer. The ob j ect_cropping_height and ob j ect_cropping_jwidth 
fields specify the height and width of the right half of the 
language selection menu . Similarly, in FIG. 28, the respective 
fields of the compos it ion__ object (2) structure together specify 

25* about 3/4 of the language selection menu from the right. In 
FIG. 29, the respective fields of the composition^ object (3) 
structure together specify the entire part of the language 
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selection menu. By rendering the cropped part of the graphics 
object as shown in FIGs . 27, 28, and 29 to the graphics plane, 

« 

the right half of the language selection menu, the 3/4 of the 
language selection menu from the right, and the entire language 
5 selection menu are overlaid on the video. 

As described above, according to the third embodiment of 
the present invention, with the compos it ion_obj ect structures, 
an animation effect is easily defined as desired. For example, 
it is easily defined that a graphics object stored on the object 
10 buffer is rendered so as to make a gradual appearance on screen 
or gradual disappearance from the screen. 

(FOURTH EMBODIMENT) 

A fourth embodiment of the present invention relates to 
15 an improvement made through optimizing the order of ODSs within 
a DS . As described above, the ODSs within a DS are referenced 
by the ■ interactive_composltion so as to present an in—effect 
or out-effect animation or one state of a button. The order 
of ODSs within a DS is determined depending on whether the ODSs 
20 are used for an in-ef f ect animation, an out effect animation, 
or a button state. 

Specifically, the ODSs within a DS are grouped into (1) 
ODSs used for presentation of an in-ef feet animation for Page [0] , 
(2) ODSs used for presentation of the normal state of a button, 
25 (3) ODSs used for presentation of the selected state of a button, 
( 4 ) ODSs used for presentation of the activated state of a button, 
(5) ODSs used for presentation of an in-effect for Page [0] and 
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an in- effect and out-effect animations for Page [1] onward. That 
is to say, ODSs associated with the same presentation is put 
into the same group. The group of ODSs associated with the 
in-effect is referred to as an "in-effect group". The group 
5 of ODSs associated with each button state is referred to as a 
"button-state group". The groups of ODSs associated with the 
out-effect of Page[0], in-ef fects and out-effects of Page[l] 
onward are referred to as a "remain group". 

Those groups of ODSs are arranged in the following order: 
10 in-effect group -> normal-state group -> seleted-state group -> 
activated-state group remain group. In this way, the order 
of ODSs within a DS is determinedbased on the presentation targets 
with which the ODSs are associated. 

Next, a description is given specifically to the order of 
15 ODSs within a DS n . FIG. 30 is a view showing the order of ODSs 
within the DS n . 

In the figure, ODSi-ODS g belong to a group of ODSs associated 
with the in-effect for Page[0] (in-effect group). 

ODSg+i-ODSh are a group of ODSs associated with the normal 
20 state of buttons (ODSs for Normal state) . 

ODSh+i-ODSi belong to a group of ODSs associated with the 
selected state of the buttons (ODSs for Selected state) . 

ODSi+i-ODSj belong to a group of ODSs associated with the 
activated state of the buttons (ODSs for Activated state) . 
25 ODSj+i-ODSk belong to a group of ODSs" associated with the 

out-effect of Page[0] and ODSs associated with the in-ef fects 
and out-effects of pages from Page[l] onward (remain group). 
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The groups of ODSs are arranged in the following order: 
in— effect group -> normal-state group -> selected-state group -> 
activated-state groups remain group. This order is determined 
so that ODSs needed for the first presentation of interactive 
display are read first, whereas ODSs needed only after a display 
update are read later. 

Next, a description is given to the order of ODSs when there 
is a multiple-reference. The term "multiple-reference" used 
herein means that the same object_id is referenced by two or 
more pieces of normal_state_inf o, selected_state__inf o, and 
activated__state__inf o within an ICS. For example, with the 
multiple references, a specific graphics object used to render 
a button in the normal state is commonly used to render another 
button in the selected state . In other words, the graphics ob j ect 
is shared, so that the number of ODSs can be reduced. Here, 
a problem arises as to which button-state group an ODS having 
the multiple-reference belongs . 

Specifically, when one ODS is associated with one button 
in the normal state as well as another button in the selected 
state, it needs to be determined whether the ODS belongs to the 
button-state group for the normal state or the button-state group 
for the selected state. 

In such a case, the ODS placed in the button-state group 
that first appears in the ICS. 

For example, if an ODS is referenced by both th 3 e normal-state 
and selected-state groups, the ODS is placed in the normal-state 
group (N-ODSs) but not in the selected-state group (S-ODSs) . 
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Also, if an ODS is referenced by both the selected-state and 
activated-state groups, the ODS is placed in the selected-state 

4 

group (S-ODSs) but not in the activated-state group (A-ODSs) . 
In this manner, each ODS referenced by two or more different 
5 state_info structures, the ODS appears only once in the DS and 
placed in the first one of the button-state groups. 

The same applies to the ODSs associated with animation 
effects. If a graphics object is commonly used in the in— effect 
and the out-effect for Page[0] , the graphics object appears only 

10 once in the DS and placed in the in-effect group. Similarly, 
if a graphics ob j ect is commonly used in the in-effect for Page [0 ] 
and the in-effect or out-effect for Page [1] onward, the graphics 
object appears only once in the DS and placed in the in-effect 
group. This concludes the description of the order of ODSs where 

15 there is a multiple-reference to an ODS. 

Because of possible multiple-references to ODSs as 
described above, the DS n may not contain all the ODS groups 
mentioned above. For example, the DS n may not contain the 
selected-state group ODSs due to a multiple-reference. 

20 Furthermore, the DS n may not contain the in-effect group ODSs. 
This is also because an in-effect is not essential to the 
presentation of interactive display. 

Now, a description is given to the order of ODSs within 
the button-state group for the selected state (S-ODSs) . Among 

25 the S-ODSs, which ODSs should bte placed at the beginning depends 
on whether a default selected button is determined statically 
or dynamically. The default selected button is statically 
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determined if the def ault_selected__button_id_ref field in the 
interactive_composition is set to a valid value other than "00" . 
This valid value' specif ies the button to be selected as default. 
In the case where the def ault__selected_button_id_ref field value 
5 is valid and that ODSs associated with the specified default 
selected button are not present in the N-ODSs , the ODSs are placed 
at the beginning of S-ODSs. 

On the other hand, if the def ault__s elect ed_button_id_ref 
field is set to "00", a different button is dynamically selected 

10 as default depending on the state of playback apparatus. 

The def ault_selected_button_id_ref field is set to the 
value "00" in the case, for example,, where an AV Clip into which 
the DS is multiplexed serves as a merge point of a plurality 
of playback paths. In such a case> a different button needs 

15 to be selected as default depending on which playback path has 
been taken. Thus, it is not necessary to place specific ODSs 
at the beginning of S-ODSs. 

FIG. 31 is a view showing how the order of ODSs within S-ODSs 
differs between the cases where the 

20 def ault_selected_button_id_ref field is set to "00" and to the 
valid value specifying the button B. In the figure, arrows ssl 
indicate the order of ODSs within S-ODSs in the case where the 
def ault_selected__button_id__ref field specifies the button_id 
identifying the button B. Arrows ss2 indicate the order of ODSs 

25 within S-ODSs in the case where the 

def ault_selected_button_id__ref field is set to "00". As shown 
in the figure, in the * case where the 
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def ault__selected_button_id_ref field specifies the button B, 
the ODSs associated with the selected state of button B (ODSs 

4 

for Button B) are placed at the beginning of the S-ODSs, followed 
by the ODSs associated with the other buttons (ODSs for Buttons 
5 A, C, and D) . On the other hand, in the case where the 
default selected button id ref field is set to "00", the ODSs 
associated with the selected state of button A are placed at 
the, beginning of the S-ODSs . As described above, the order of 
ODSs within S-ODSs differs depending on whether the 
10 def ault__selected__button_id__ref field value is valued or not. 

According to the fourth embodiment, the ODSs within an DS 
are arranged in the order of appearance in the DS . Thus, the 
display composition containing a number of graphics elements 
can be smoothly presented. 

15 

(FIFTH EMBODIMENT) 

In the first embodiment, how to allocate DSs to the time 
axis of AV Clip playback is described. In a fifth embodiment 
of the present invention, a description is given to how to set 
20 PTS and DTS values. 

First of all, a description is given to a mechanism of 
effecting synchronous control based on the DTS and PTS within 
an ODS. 

The DTS shows a time, with the time accuracy of 90 KHz, 
* 25 at which decoding of the associated ODS needs to be started. 
The PTS shows a deadline for completing the decoding. 

The decoding of ODS can not be completed instantly, and 
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takes a certain duration. In order to explicitly show the start 
and end times of decoding the ODS , the DTS and PTS show the decoding 

■ 

start time and decoding deadline. 

The PTS value shows the deadline by which decoding of the 
5 associated ODS needs to be completed and the resulting 
uncompressed graphics object-needs to be available on the object 
buffer of the playback apparatus. 

The decoding start time of an arbitrary ODSj belonging to 
a DS n is shown by DTS(DS n [ODS]) with the time accuracy of 90 
10 KHz. Thus, the decoding deadline of the ODSj is determined by 
a sum of the DTS(DS n [ODS] ) value and a maximum time which may 
be taken for the decoding. 

Here, let SIZE (ODS n [ODSj ] ) denotes the size of the ODSj after 
decompression, and Rd denotes the decoding rate for ODSj, then 
15 the maximum time taken for the decoding is ceil ( SIZE ( DS n [ODSj ] ) ) . 

By converting this maximum time to the accuracy of 90 KHz 
and adding the result to the DTS of the ODSj , the decoding deadline 
to be shown by the PTS is calculated with the accuracy of 90 
KHz . 

20 The PTS of the ODSj belonging to the DS n can be expressed 

by the following equation. 

PTS (DS [ODSj] ) = DTS (DS n [ODSj] ) 

+ 90, 000 x ceil (SIZE (DS n [ODSj] ) /Rd) 

25 

In addition, the relation between two adjacent ODSs (ODSj 
and ODSj + i) needs to satisfy the following equation. 
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PTS (DS n [ODSj] ) < DTS (DS n tODS j+ i] ) 

This concludes the description of the PTS and the DTS of 
5 an ODS. Next, a description is given to the PTS within an ICS. 
The ICS needs to be loaded to the composition buf f er of the playback 
apparatus before the decoding start time of the first ODS within 
the DS n (i.e. before the time shown by DTS (DS n [ODSi] ) ) and also 
before the first presentation of theDS n becomes valid (i.e. 
10 the time shown by PTS ( DS n [ PDSi] ) ) . Thus , the following equations 

* 

need to be satisfied. 

DTS (DS n [ICS] ) — DTS ( DS n [ODSi] ) 
DTS ( DS n [ ICS ] ) ^ PTS (DS n [PDSi] ) 

15 

Next, a description is given to the PTS within the ICS of 
the DS n . The PTS (DS n [ICS] ) value is set to satisfy the following 
equation . 

20 PTS (DS n [ICS] ) ^ PTS (DSn [ICS] ) 

+ DECODEDURATION + T RAN S FERDURAT I ON 

The DECODEDURATION shows the time taken for decoding 
graphics obj ects needed for the first presentation of the DS n (ICS) . 
25 The decoding start time is equal to the DTS (ICS) value at the 
earliest. Here, let LASTODSPTS denote the PTS associated with 
the graphics object of which decoding is completed at the latest 

58 



WO 2005/048261 



PCT/JP2004/017215 



among graphics objects needed for the first presentation of 
DS n (ICS), then the DECODEDURATION is equal to LASTODSPTS ( DS n ) 

- DTS (DS n [ICS] ) • 

The LASTODSPTS value takes on a different value depending 
on the def ault_selected_button_id_ref setting and the order of 
ODSs within the DS . FIGS. 32A, 32B, 33A, 33B, and 33C show 
different ODS orders to illustrate LASTODSPTS setting using the 
DTS of an ODS. 

FIG. 32A shows the case where the default selected button 
is statically determined,. i.e. where the 

def ault_selected__button_id_ref field is set to a non-zero value . 
In this case, the LASTODSPTS takes on the PTS value of the first 
ODS within the S-ODSs (i.e. the value of PTS ( DS n [ODS h+1 ] ) ) . In 
the figure, the ODS h +i is enclosed within the solid line box. 

FIG. 32B shows the case where the default selected button 
is determined dynamically, i.e. where the 

def ault_selected__button_id__ref is set to "00". In this case, 
the LASTODSPTS takes on the PTS value of the last ODS among the 
S-ODSs (i.e. the value of PTS (DS n [ODSjJ ) ) . In the figure, the 
ODSi is enclosed within the solid line box. 

FIG. 33A shows a DS that includes no ODS associated with 
the selected state. In this case, the LASTODSPTS takes on the 
PTS value of the last ODS among the N-ODSs (i.e. the value of 
PTS (DS n [ODS h ] ) ) • In the figure, the ODS n is'enclosed within the 
solid line box. 

FIG. 33B shows a DS that includes no ODS associated with 
the normal state. In this case, the LASTODSPTS takes on the 
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PTS value of the last ODS among the in-effect ODSs (i.e. the 

* 

value of PTS (DS n [ODSg] ) ) . In the figure, the ODS g is enclosed 
within the solid line box. 

FIG. 33C shows a DS that includes no ODS at all. In this 
5 case, the LASTODSPTS takes on the DTS value specified in the 
ICS of the DS n (i.e. the value of DTS (DS n [ICS] ) ) . In the figure, 
the ICS is enclosed within the solid line box. 

Note that the above description applies to the case where 
the DS n is not an Epoch Start DS . If the DS n is an Epoch Start 

10 DS, a different description is applied. 

In the case of an Epoch Start DS, the graphics plane may 
be cleared entirely. The time taken for clearing the graphics 
plane may be longer than the time calculated by LASTODSPTS (DS n ) 
- DTS (DS n [ICS] ) - According to a target decoder model for the 

15 BD-ROM, the graphics decoding and the graphics plane clearing 
are performed by different processing units (the graphics 
processor performs the decoding) . Thus, the graphics decoding 
and the plane clearing are performed in parallel. Even if the 
decoding is completed for graphics data necessary for 

20 presentation of the first display composition, the graphics date 
can not be rendered to the graphics plane unless the plane has 
been cleared. Thus, the DECODEDURATION needs to set to the value 
equal to the plane clear duration. Suppose the graphics plane 
has the width and height equal to the video__width and video_height 

25 field values, respectively. Furthermore, the transfer rate 
between the object buffer and the graphics plane is 128 Mbps. 
In order to clear the entire graphics plane, the playback 
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apparatus is required to overwrite the graphics plane having 
the size of 8 x video_width x video_height at the transfer rate 
of 12 8 Mbps . Expressing the duration taken for this overwriting 
in90KHz, the time taken for clearing the graphics plane (referred 
to as "PLANECLEARTIME") is calculated in the following equation. 

PLANECLEARTIME = ceil ((90000 x 8 x DS n [ ICS ] . video_width 

x DS n [ICS] . video_height) /12 8, 00 0, 0 00) 

FIG . 3 4A shows an algorithm for obtaining the DECODE DURAT I ON 
value from the value calculated by LASTODSPTS (DS n ) 
DTS(DS n [ICS] ) and the value of PLANECLERATIME. In the algorithm, 

■ i 

the larger one of the LASTODSPTS (DS n ) - DTS (DS n [ICS] ) value and 
the PLANECLERATIME value is determined as the DECODE DURAT ION 
value. 

As shown in the figure r if the composition_state field of 
the DS n (ICS) indicates that the DS n is not an epoch start DS 

(if (DS n (ICS) . composition_state==EPOCH_START) else) , the value 
of LASTODSPTS (DS n ) - DTS (DS n [ICS] ) is determined as the 
DECODEDURATION value (return (LASTODSPTS (DS n ) - DTS (DS n [ ICS ] ) ) ) . 
On the other hand, if the composition__state field of the DS n 

(.ICS) indicates that the DS n is an epoch start DS 

(if (DS n (ICS) . composition_state — EPOCH START) ) , the larger one 
of the LASTODSPTS (DS n ) ~ DTS (DS n [ICS] ) value and the 
PLANECLERATIME value is determined as the DECODEDURATION value 

( return (max (LASTODSPTS (DSJ - DTS (DS n [ICS] ) ))) . 

FIGs- 34B and 34C are views showing examples of 
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DECODEDURATION setting depending on whether or not the 
PLANECLEART IME value is larger than the LASTODSPTS ( DS n ) - 
DTS(DS n [ICS] ) value. If the PLANECLEART IME value is larger than 
the LASTODSPTS (DS n ) " DTS (DS n [ICS] ) value, the PLANECLEART IME 
is determined as the DECODEDURATION value, as shown in FIG. 34B. 
On the other hand, if the PLANECLEART IME value is smaller than 
the LASTODSPTS (DS n ) ~ DTS (DS n [ICS] ) value, the LASTODSPTS ( DS n ) 
- DTS (DS n [ICS] ) value is determined as the DECODEDURATION value, 
as shown in FIG . 34C. 

The TRANS FERDU RAT ION shows the time taken for transferring 
graphics data needed for presentation of the first display 
composition from the object buffer to the graphics plane. The 
first display composition within a DS differs depending on 
whether an in-ef f ect is provided for the DS and whether the default 
selected button is determined statically or dynamically. Thus, 
the TRANS FERDURATION also differs depending on the first display 
composition. If the first display composition is associated 
with an in-ef feet, the time taken for transferring the 
uncompressed graphics needed for the presentation of the 
in-ef feet is determined as the TRANS FERDURATION . 

Here, the in-ef feet defined by the DS n [ICS] is for 
introducing Page[0] and rendered in windows of the number that 
is indicated by the number__of_window field value. Thus, the 
total size of graphics data needed for the first display 
composition of the in-effect is equal to the total size of the 
windows (0) through (number_of_windows - 1 ) . Thus, to calculate 
the time taken for transferring graphics data needed for 
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presentation of the first display composition within the 
in-ef feet sequence (EFFECTTD) , the total window size is divided 
by the transfer rate, of the playback apparatus (12 8 Mbps) . To 
express the result in the time accuracy of the PTS (90 KHz) , 
5 the result is further multiplexed by 90 KHz. The equation for 
. calculating the EFFECTTD (DS n ). is shown in FIG. 35A. 

On the other hand, if the ■ DS n is not provided with any 
in-ef feet, the first display composition for presentation of 
the DS n [ICS] is Page[0] of the Multi-Page menu. If the default 

10 selected button is statically specif ied, Page[0] of the 
Multi-Page menu is presented as soon as uncompressed data needed 
for the selected state of the default selected button and the 
normal state of the other buttons is transferred from the object 
buffer to the graphics plane. 

15 Here, Page[0] of the Multi-Page menu associated with the 

i 

DS n (ICS) includes the buttons of the number indicated by the 
number__of_button field value. Thus, the data size needed for 
presentation of the first display composition is calculated as 
follows. First, the total size is calculated for the graphics 

20 data associated with the normal state of all the buttons on the 
page, which are specified by DS n ( ICS) . Page [0] . Buttons from ( 0 ) 
through (number_of_Button-l ) . Then, to the thus calculated 
total size, the size of graphics data associated with the selected 
state of the default selected button of Page[0] (SBSIZE (DS n , 

25 DS n [ICS] ,PAGE[0] . def ault_selectedjoutton_id_ref ) ) is added. 
Further, from the resulting value, the size of graphics data 
associated with the normal state of the default selected button 
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(NBSIZE (DS n , DS n [ICS] . PAGE[0] . def ault_selected_button_id_ref 
) ) is subtracted to obtain the data size needed for presentation 
of the first display composition. 

The thus calculated data size needed for presentation of 
the first display composition is divided by the transfer rate 
of the playback apparatus (128 Mbps\, and multiplied by 90 KHz 
for expressing the result in the time accuracy of PTS (90 KHz) . 
Then, the transfer duration taken for presentation of the first 
page is calculated. FIG. 35B shows the equation for calculating 
the PAGDEFAULTTD (DS n ) in the case where no in-effect is provided 
and the default selected button is statically determined. 

In the case where the default selected button is dynamically 
determined, the equation shown in FIG. 35B is not applicable 
because it is not known which of the buttons on Page [ 0 ] is selected 
as default. Thus, the TRANSFERDURATION needs to be calculated 
for the worst case. The worst case is where the button with 
the largest size is selected as default among the buttons defined 
by the DS n (ICS) . Page [0] . Among the buttons of DS n (ICS) . Page [0] , 
the button with the largest size is denoted as LRG{button: 

button e DS n [ ICS] . PAGE [ 0 ]. Button} . ' 

Then, the data size needed to be transfer in the 
above-mentioned worst case (the maximum data size) is calculated 
as follows. First, the total size of all the buttons on Page[0] 
(Page [0] .Buttons from (0) through (number_of_Button-l) ) is 
calculated. To the thus calculated total size, the size of the 

button LRG (BSIZE (DS n , LRG{button: button e 
DS n [ICS] . PAGE [0] . Button} ) ) is added. Further, from the 
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resulting value, the size of graphics data associated with the 
normal state of the button LRG (NBSIZE (DS n/ LRG{button: button 

g DS n [ ICS ] . PAGE [ 0 ] . Button } ) ) is subtracted to obtain the maximum 
data size. 

The maximum data size calculated in the above manner is 
dividedby the transfer rate of the playback apparatus ( 128 Mbps) , 
and further multiplied by 90 KHz for expressing the result in 
the time accuracy of PTS (90 KHz) . Then, the transfer duration 
taken for presentation of the first page of the Multi-Page menu 
is calculated. FIG. 35C shows the equation for calculating the 
PAGENODEFAULTTD (DS n ) in the case where no in-effect is provided 
and the default selected button is dynamically determined. 

FIG. 36 is a view showing an algorithm for selectively 
determining one of the EFFECTTD, PAGEDEFAULTTD, and 
PAGENODEFAULTTD as the TRANS FERDURAT ION . As shown in- the figure , 
if at least one in-effect is provided (if 
(DS n [ICS] . PAGE[0] . IN__EFFECTS . number_of_effects ! = 0) ) , the 
EFFECTTD is determined as TRANS FERDURAT I ON {return 
EFFECTTD (DS n ) ) . If no in-effect is provided for introducing 
Page[0] and no default selected button is statically specified 
(DS n [ICS. PAGE [0] - default_selected_button_id_ref OxFFFF] ) , 
the PAGEDEFAULTTD is determined as the TRANS FERDURATION (return 
PAGENODEFAULTTD (DS n ) ) . If no in-effect is provided for 
introducing Page [0] andthe default selectedbutton is statically 
determined, the PAGEDEFAULTTD is determined as the 
T RAN S FE RDU RAT I ON (return PAGEDEFAULTTD (DSn) ) . 

As described above, according to the fifth embodiment of 
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the present invention, the DTS and PTS values of an ICS are 
optimally determined in accordance with the data size of ODSs. 
Thus, the interactive control is smoothly performed without 
delay. 

5 

(SIXTH EMBODIMENT) 

A sixth embodiment of the present invention relates to the 
playback apparatus used for playback of the BD-ROMs described 
in the first to fifth embodiments. FIG. 37 is a view showing 

10 the internal structure of the playback apparatus , The playback 
apparatus according to the sixth embodiment is industrially 
manufactured based on the internal structure shown in FIG . 37. 
The playback apparatus is mainly composed up of" two parts that 
are a system LSI and a drive device. The playback apparatus 

15 can be industrially manufactured by mounting these parts on a 

r 

cabinet and substrate of the apparatus. The system LSI is an 
integrated circuit including various processing units for 
achieving the functions of the playback apparatus. 

The playback apparatus manufactured in the above manner 

20 is composed of a BD drive 1, a read buffer 2, a demultiplexer 
(De-MUX) 3, a video decoder 4 , a video plane 5 , P-Graphics decoder 
6, a presentation plane 7, an adder 8, an I-Graphics decoder 
9, an interactive graphics plane 10, an adder 11 , an audio decoder 
13, CLUT (Color Lookup Table) units 14 and 15, a PRS set 16, 

25 a system clock 17, a userMzimeout timer 18, a selection__timeout 
timer 19, a compostion_timeout timer 20 , an effect__duration timer 
21, an operation receiving unit 22, and a state control unit 
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23. 

The BD drive 1 performs loading, reading, and ejecting of 

4 

the BD-ROM. The BD drive .1 makes access to the BD-ROM 100. 

The read buffer is a FIFO (first-in first-out) memory. 
Accordingly, TS packets read from the BD-ROM are removed from 
the read buffer 2 in the same order as they arrive. 

The De-MUX 3 reads TS packets from the read buffer 2 and 
converts them to PES packets. Among the resulting PES-packets, 
PES packets having a predetermined stream PID are outputted to 
one of the video decoder 4, the audio decoder 13, the P-Graphics 
decoder 6, and the I-Graphics decoder 9. 

The video decoder 4 decrypts PES packets received from the 
De-MUX 3, and rendered the resulting uncompressed-picture data 
to the video plane 5. 

The video plane 5 is a plane memory for storing uncompressed 
picture data. The term "plane" refers to a memory area of a 
playback apparatus for storing one screen of pixel data. With 
provision of multiple planes, the playback apparatus can add 
the stored data on each plane pixel by pixel to produce one video 
output, whereby a plurality of graphics images are overlaid as 
one composite image. The resolution of the video plane 5 is 
1920 x 1080. Picture data is stored on the video plane 5 as 
16-bit pixel data expressed in YUV values. 

The P-Graphics decoder 6 decodes a graphics stream read 
from the BD-ROM or HD, and renders the resulting uncompressed 
graphics to the presentation graphics plane 7. As the graphics 
stream is decoded and rendered, subtitles appear on screen. 
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The presentation graphics plane 7 is a memory having an 
area for storing one screen of uncompressed graphics. The 
resolution of this plane is 1920 x 1080, and uncompressed graphics 
are stored on the presentation graphics plane 7 as 8-bit pixel 
5 data expressed in index color. By converting the index color 
using the GLUT (Color Lookup Table) , the uncompressed graphics 
stored on the presentation graphics plane 7 are presented. 

The adder 8 overlays the uncompressed picture data (i) with 
the contents stored on the presentation graphics plane 7 . 
10 The I-Graphics decoder 9 decodes an IG stream read from 

theBD-ROMorHD, and renders the resulting uncompressed graphics 
to the interactive graphics plane 10. 

To the interactive graphics plane 10 , uncompressed graphics 
data decoded by the I-Graphics decoder 9 is rendered. 
15 The adder 11 overlays the data on the interactive graphics 

< 

plane 10 with the output of the adder 8 (i.e. with the interim 
video data maid by overlaying uncompressed picture data and the 
contents stored on the presentation graphics plane 7) to make 
the final video output . 
20 The audio decoder 13 decodes PES packets received from the 

De-MUX 3 to output uncompressed audio data. 

The CLUT unit 14 converts index colors of the uncompressed 
graphics stored on the presentation graphics plane 7 to Y, Cr, 
and Cb values. 

25 The CLUT unit 15 converts index colors of the uncompressed 

graphics stored on the interactive graphics plane 10 to Y, Cr-, 
and Cb values. 
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The PSR set 16 is a set of registers provided within the 
playback apparatus. The set of registers include 64 player 

* 

status registers (PSRs) and 4,096 general purpose registers 
(GPRs) . The 64 player status registers represent the current 
5 status of the. playback apparatus. Specifically, the PSRs 5-8 
represent the current playback point. The PRS 8 takes on a 
value ranging from 0 to OxFFFFFFFF representing the current 
playback point (current PTM: Presentation Time) in 45 KHz. The 
PSR 11 represents the page_id of the currently active page 
10 (current page ) . The PSR 10 represents the button in the selected 
state (current button) on the current page. 

The system clock 17 generates clock signals. The De-MUX 
3 performs the conversion of PES packet in synchronism with the 
clock signals. 

15 The user_timeout timer 18 is set to the - value of 

user_time_out_duration field at the time specified by the 
PTS (DSn[ICS] ) . When the user_time__out_duration field value is 
set, the timer 18 starts to count down toward zero, in system 
clock ticks. The timer 18 is reset back to the 

20 user_time_jout_duration field value each time a user operation 
is received. If no user operation is received before the timer 
18 reaches zero, the timer 18 times out. 

The selectionjtimeout timer 19 is set to the value of 
selection_time__out__pts field at the time specified by the 

25 PTS (DS n [ICS] ) . When the select ion_time_out_pts field value is 
set, the timer 19 starts to count down toward zero in system 
clock ticks. The timer 19 is reset back to the 

» 
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selection_time_out_pts field value each time a user operation 
is received. If no user operation is received before the timer 

« 

19 reaches zero, the timer 19 times out.* When the timer 19 times 
out, it is known that the valid interaction period shown in FIG. 
5 9 has ended. 

The composition_timeout timer 20 is set to the value of 
compost ion_time_out_pts field at the time specified by the 
DTS (DS n [ICS] ) . When the compostion_time__out_pts f ield value 
is set, the timer 20 starts to count down toward zero in system 

10 clock ticks. The timer 20 is not reset even if a user operation 
is received. When the timer 20 reaches to zero, i.e. the timer 
2 0 times out, it is known that the Epoch END has been reached. 

The ef f ect_duration timer 21 is set to the value of 
effect duration field at the time specified by the DTS (DS n [ICS] ) 

15 if the DS n is an Epoch Start DS. When the effect_duration field 
value is set, the timer 21 starts to count down toward zero in 
system clock ticks. The timer 21 is not reset even if a user 
operation is received. When the timer 21 reaches zero, i.e. 
the timer 21 times out, it is known a next display composition 

20 of the animation effect needs to be displayed. 

The operation receiving unit 22 receives a user operation 
made on the remote controller or the front panel of the playback 
apparatus, and outputs information representing the received 
user operation (the information is hereinafter referred to as 

25 "UO") to the I-Graphics decoder 9. 

The state control unit 23 sets desired values to the PSRs 
according to the instructions from the I-Graphics decoder 9. 
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Such instructions may £>e given by way of (i) direct addressing 
or (ii) indirect addressing. In the case of direct addressing, 
the I-Graphics decoder 9 outputs an immediate value to be set 
to an appropriate register of the PSR set 16. In the case of 
indirect addressing, the I— Graphics decoder 9 outputs a 
notification of a change in the playback apparatus status or 
in the user preferences. On receiving such a notification, the 
status controller 23 determines the value to be set to reflect 
the change, and sets the thus determined value to an appropriate 
register of the PSR set 16. The status controller 23 determines 
the value by performing a "procedure when . playback condition 
is changed" or "procedure when change is requested". 
Hereinafter, a description is given to the procedures performed 
for setting the PSR 11 (current page) and the PSR 10 (current 
button) . 

FIG. 38A is a flowchart of the "procedure when playback 
condition is changed", preformed with respect to the PSR 11. 
This procedure is performed to set the first page_inf o structure 
within an ICS to the PSR 11 (step S99) . 

FIG. 38B is a flowchart of the "procedure when change is 
requested", preformed with respect to the PSR 11. Suppose that 
a user operation is received and a page having the page number 
X is requested to be presented. In response to the user operation, 
the status controller 23 judges whether X is a valid value (step 
S100) . If X is a valid value, the PSR^ll is set to the value 
of X (step S101) . If X is invalid, the PSR 11 remains* unchanged 
(step S102) . 
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The value of PSR 11 changes as described above. Turning 
now to the PSR 10, a description is given to the "procedure when 
playback condition is changed" and to the "procedure when change 
is requested". 

5 FIG. 3 9A is a flowchart of the "procedure when playback 

condition is changed", performed with respect to the PSR 10. 

First, the status controller 23 judges whether the 
def ault_selected_button_id_ref field value associated with the 
current page is valid (step Sill) . If the judgment in the step 
10 Sill results in YES, the def ault_selected_button__id_ref field value 
■ is set to the PSR 10 (step S112) 

If the def ault__selected_button_id__ref field value is invalid, 
it is then judged whether the PSR 10 holds a valid value (step S113) . 
If the PSR 10 is valid, the PSR 10 remains unchanged so that the currently 
15 held value is maintained (step S114) . On the other hand, if the PSR 
10 is invalid, the PSR 10 is set to the button_id_ref field value identifying 
the first button_info structure of the current page (step S115) . 

FIG. 39B is a flowchart of the "procedure when change is 
requested", performed with respect to the PSR 10. Suppose that 
20 a user operation is made and a button having the button number 
X is requested to be selected. In response to the user operation, 
the status controller 23 judges whether X is a valid button_id 
field value (step S116) . If X is a valid value, the PSR 10 is 
set to the value of X (step S118) . If X is an invalid value, 
25 the PSR 10 remains unchanged and thus the currently held value 
is maintained (stepS117) . With the above-described procedures , 
the PSRs 10 and 11 are ensured to hold current values at all 
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times. This concludes the description about the internal 
structure of playback apparatus . 

<Internal Structure of I-Graphics Decoder 9> 
5 Next, a description is given to the internal structure of 

I-Graphics decoder 9, with reference to FIG. 40. As shown in 
the figure, the I-Graphics decoder is composed of a coded data 
buffer 33, a stream graphics processor 34, an object buffer 35, 
a composition buffer 36,. and a graphics controller 37. 
10 The coded data buffer 33 is a buffer for temporally storing 

an ICS, a PDS, and an ODS along with their DTS and PTS. 

The stream graphics processor 34 decodes an ODS and outputs 
the resulting uncompressed graphics ob j ects to the ob j ect buffer 
35. 

15 The object buffer 35 is a buffer for storing a plurality 

of uncompressed graphics objects (represented by the boxes in 
the figure) , which are decoded by the stream graphics processor 
34 . 

The composition buffer 36 is a memory for storing an ICS. 
20 The composition buffer 36 supplies to the graphics controller 
37, the page_info structures provided within the ICS and the 
button_info structures provided within each page__info 
structure . 

The graphics controller 37- decodes an ICS stored in the 
25 composition buffer 37 and assembles a display composition based 
on the decoded ICS. That is, the graphics controller 37 reads 
graphics objects from the object buffer 15 with reference to 
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the page__inf o structure identif iedby the value of PSR 11 (current 
page_info structure) , and renders the read graphics objects to 
the interactive graphics plane 10. The graphics objects read 
. herein are a sequence of graphics objects specified by the 
5 normal_start_obj ect_id_ref field value through the 
normal_end_object_id_ref field value of each button_info 
structure associated with the current page. With respect to 
the button__info structure specified by the PSR 10, a sequence 
of graphics objects specified by the 

10" selected_start_object_id_ref field value through the 
selected_end_obj ect_id_ref field value is read from the object 
buffer 15. In FIG . 40, arrows bgl, bg2, bg3, and bg4 
schematically represent the rendering of graphics objects 
performed by the graphics controller 37 . As a result of the 

15 rendering, a page including the buttons 0~A, 0-B, 0-C, and O-D 
is stored on the interactive graphics plane 10 and overlaid with 
the video. 

This concludes the rough description of procedures to be 
performed by the graphics controller 37. Furthermore, the 

20 graphics controller 37 updates the display composition in 
response to the changes in the value of PSR 10 or PSR 11, the 
timeout of the timers 18-21, or a user operation received by 
the composition timeout timer 20. The procedures performed by 
the graphics controller 37 are shown in FIGs. 41-47 in detail. 

25 FIG. 41 is a flowchart of the main routine of the procedure 

performed by the graphics controller 37. In the main routine, 
steps S1-S6 forma loop. In each iteration of the loop, judgments 
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aremade as to whether specific events have occurred (steps S1-S4) , 
a display composition of the Multi-Page menu is updated to a 
subsequent display composition for animated presentation (step 
S5) , and processing is performed according to a user operation 
5 (UO processing) (step S6) . If any of the judgments in the steps 
S1-S4 results in YES , corresponding steps are performed and the 
processing returns to the main routine. 

In the step SI, the judgment is made as to whether the current 
playback point has reached the point specified by the 

10 PTS (DS n [ICS] ) . If the judgment in the step SI results in YES, 
the above described "procedure when playback condition is 
changed" is performed. As a result, Page [0], which is the 1 st 
page of the Multi-Page menu, is set as the current page (step 
S7). Then, the user__timeout timer 18, the selection_timeout 

15 timer 19, and the composition_timeout timer 20 are- all reset 
(step S8) , and the current page is presented on screen (step 
S9) . The processing then goes back to the step SI for the next 
loop iteration. 

In the step S2, the judgment is made as to whether the 

20 user_timeout timer 18 has timed out. If the judgment in the 
step S2 results in YES, a loop of steps S11-S14 is performed. 
In each iteration of the loop, an arbitrary Page[j] of the 
Multi-Page menu is removed from the screen (step S13) , followed 
by playback of an out-effect animation provided for Page [ j ] (step 

25 S14) . The loop is repeated for each page from Page[0] onward, 
defined within the ICS (steps Sll and S12) . 

In the step S3, the judgment is made as to whether the 
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selection_timeout timer 19 has timed out. 1 If the judgment in 
the step S3 results in YES, a button on the current page is- 

4 

automatically activated (auto-activate: steps S10) . Then, the 
• processing returns the step SI for the next loop iteration. 
5 In the step S4, the judgment is made as to whether the 

composition_timeout timer 2 0 has timed out. If the judgment 
in the step S4 results in YES, Page[0] , which is the first page 
of the Multi-Page menu, is removed from the screen (step S15) , 
followed by playback of the out-effect animation provided for 

10 Page[0] (step S16) . Then, the processing returns to the step 
SI for the next loop iteration. 

Now, a description is given to the first presentation of 
the multi-page menu. The first presentation is carried out 
through steps S15-S23 shown in FIG. 42, after the state control 

15 information 23 performs the above-described "procedure when 
playback condition is changed" to set Page[0] as the current 
page. In the step S24, the in-effect animation for the current 
page is executed. In the step S25 that follows, a current button 
setting procedure is performed . The current button is specified 

20 bythePSRIO. Thus, in the step S25, the "procedure whenplayback 
condition is changed" is performed' with respect to the PSR 10, 
so that the current button is determined. Then, the processing 
moves onto the steps S17-S22. 

The steps S17 and S22 form a loop which is repeated for 

25 each button_info structure provided in the current pag^e (steps 
S17 and S18) . The button__info structure to be processed in the 
current iteration of the loop is designated as button info (p) . 
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In the step S19, a judgment is made as to whether the 
button_info (p) is associated with the current button. If the 
judgment in the step S19 results in YES, the step S20 is performed 
next. If not, the step S21 is performed next. 
5 In the step S21, among the graphics objects on the object 

buffer 35, one specif iedby the normal_start_ob j ect_id_ref field 
value is designated as the graphics object (p) . 

In the step S2 0, among the graphics objects on the object 
buffer 35, one specified by the selected_start_ob j ect__id__ref 

10 field value is designated as the graphics object (p) . 

Once the graphics object (p) is designated in the step S20 
or S21, the graphics object (p) is rendered to the interactive 
graphics plane 10 at the position specified by the 
button_horizontal__position and button__vertical_position field 

15 values (step S22). By repeating the above steps for each 
button_info structure's provided in the current page, among a 
plurality of graphics obj ects associated with each button state, 
the first graphics object is rendered for each button to the 
interactive graphics plane 10 . After repeating the above steps , 

20 the CLUT unit 15 is set so as to present the current page using 
the pallet data specified by the pallet_id_ref field value of 
the current page (step S23) . This concludes the description 
of the flowchart shown in FIG. 42. 

FIG. 43 is a flowchart of the procedure for playback of 

25 an in~effect animation. In the flowchart, a variable t denotes 

V 

an arbitrary one of display compositions within the 
effect_sequence defined for the in-effect. In addition, a 
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variable u denotes an arbitrary one of a composition_ob j ect out 
of composition_obj ects used for the effect (t) . As shown in the 
flowchart, first of all, the variables t and u are initialized 
• to "0" (step S30) . Then, the ef f ect_duration_timer 21 is set 
5 to the value of the effect ( t ) . effect_duration field ( step S31 ) , 

i 

and the CLUT units 14 and 15 are set so -as to use the pallet 
data identified by'the effect (t) . pallet_id__ref field value for 
presentation (step S32) . Then, a loop of steps S33-S40 is 
performed. The loop is a two-level loop. The first-level loop 

10 ( steps S33-S37 ) uses the variable u as a control variable, whereas 
the second-level loop (step S33-S40) uses the variable t as a 
control variable. 

In the first-level loop, a judgment is made as to whether 
the ob j ect__cropped_f lag field provided within the 

15 composition_object (u) structure associated with the effect (t) 
is set to "0" (step S33) . If the field is set to x% 0", no graphics 
object is presented on screen for composition_ob j ect (u) (step 
S35) . If the field is set to "1", on the other hand, the object 
is cropped according ' to the 

20 ob j ect__cropping_horizontal_jposition, 

ob j ect_cropping_vertical_position, cropping__width, and 
cropping_height fields of the composition_ob j ect (u) . The 
cropped part of the graphics object is then rendered within the 
window identified by the window__id_ref field of the 

25 composition__obj ect (u) , at the position specified by the 
composition_ob j ect__horizontal__position and ' the 

composition__obj ect vertical_jposition fields within the 
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composition__ob j ect (u) (step S34). Then, the variable u is 
incremented by "1" . The first-level loop is repeated until the 
variable u is equal to the number_of_composition__ob j ect field 
• value . 

5 In the second-level loop, for each iteration of the 

first-level loop (steps S33-S37), s the following steps are 
performed. After the ef f ect__duration times out (step S38) , the 
variable t is incremented by "1" , and the variable u is initialized 
to XN 0" (steps S3 9) . The second-level loop is repeated until 

10 the variable t is equal to the number_of_effect (t) field value 
(step S40) . This concludes the description of the procedure 
for playback of In-effect. Note that no description is given 
to the procedure for playback of an out-effect, because it is 
basically the same to the above-described procedure. 

15 Immediately after playback of the in-effect, a current page 

is presented and the first display is updated for animated 
presentation . The first display composition of the current page 
is presented by rendering to the interactive graphics plane 10, 
the graphics objects associated with the 

20 normal_start_obj ect_id_ref and the 

selected_start_obj ect_id__ref field values provided in each 

* 

button_info structure. Each button is presented in animation 
by updating the button image on the interactive graphics plane 
10 for each iteration of the loop in the main routine. Each 
25 button image is updated by overwriting the interactive graphics 
plane 10 with an arbitrary one of a sequence of graphics objects 
(gth graphics object) associated with the button. That is to 
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say, each button is presented in animation by rendering a sequence v 
of graphics objects associated with the normal_state_inf o and 
selected_state_inf o fields provided within each button_info 
- structure, one by one to the interactive graphics plane 10 . Note 
that the variable qls used to specify individual graphics obj ects 
associated with the normal__state_inf o and selected_state_inf o 
fields provided within each button_info structure. 

Now, with reference to' FIG. 44, a description is given to 
the procedure for display update for animated presentation. 

In a step S41, a judgment is made as to whether the first 
display composition has been presented. If the first display 
composition is not yet presented, the processing returns to the 
main routine without performing any operation. If the first 
display composition has been presented, on the other hand, steps 
S42-S53 are performed. The steps S41-S55 form a loop in which 
the steps S44-S55 are repeated for each button_info structure 
in the ICS (steps S42 and S43) . 

In the step S4 4 , the variable gis set to the value of variable 
animation (p) corresponding to the button__info (p) structures As 
a result, the variable q represents the current frame number 
of the button_inf o (p) . 

In the step S4 5, a judgment is made as to whether the 
button__inf o (p) corresponds to the button currently in the 
selected state (i.e. the current button). 

If the button_inf o (p) does not correspond to the current 
button, the variable <j is added to the norraal_start_ob j ect__id_ref 
in the button_inf o (p) to obtain an ID(q) (step S46) . 
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If the button_inf o (p) corresponds to the current button, 
the step S47 is performed. 

In the step S 47 , a j udgment is made as to whether the current 
button is in the activated state. If the judgment in the step 
5 S47 results in YES, the variable q is added to the 
activated_start_ob j ect__id_ref field value in the 
button_inf o (p) to obtain an ID(q) (step S54) . Then, one of the 
navigation commands associated with the button_inf o (p) is 
executed (step S55) . 

10 On the other hand, if the current button is not in the 

activated state, the variable q is added to the 
selected__start_ob j ect_id_ref field value in the button_inf o (p) 
to obtain an ID(q) (step S48). 

Once the ID(q) is obtained, among the graphics objects 

15 stored on the object buffer 35, the one identified by the ID(q) 
is rendered to the interactive graphics plane 10 . The rendering 
position on the interactive graphics plane 10 is specified by 
the button_horizontal_jposition and the 

button_vertical_j?osition fields provided in the button info (p) . 

20 Through the loop iteration, the g th one of the graphics 

objects associated with the selected (or activated) state of 
the current button is rendered to the interactive graphics plane 
10 . 

In the step S50, a judgment is made as to whether the sum 
25 of the normal__start_obj ect__id_ref field value and the variable 
ghas reached the normal_end_ob j ect_id_ref field value. If the 
sum has not reached the normal end object id ref field value, 
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the variable q is incremented by "1" and the incremented variable 
q is set as the variable animation (p) (step S51) . On the other 

* 

hand, if the sum has reached the normal_end_obj ect_id_ref field 
•value, a judgment is made as to whether the repeat_flag field 
5 is set to "1" (step S52) . If the repeat_flag field is set to 
"1", the variable animation (p) is initialized to "0" (step S53) . 
The above steps are repeated for each button_info structure 
within the ICS (steps S42 and S43) , and then processing returns 
to the main routine. 

10 Through the above-described steps of S41-S53, each button 

image presented on screen is updated with a new graphics object 
each time the main routine is performed. Thus, by repeating 
the main routine, the buttons images are presented in animation. 
Whenpresenting the buttons in animation, the graphics controller 

15 37 adjusts the duration for presenting each graphics object so 
as to maintain the animation_f rame_rate_code . This concludes 
the description of the procedure for animated presentation. 

In the step S47 described above, if it is judged that the 
current button is in the activated state, a navigations command 

20 associated with the current button needs to be executed in the 
step S55. FIG. 45 is a flowchart of the procedure for executing 
a navigation command. First, a judgment is made as to whether 
the navigation command is a SetButtonPageCommand (step S56) . 
If the navigation command is not a SetButtonPageCommand, the 

25 navigation command is simply executed (step S57) . On the other 
hand, if the navigation command is "a SetButtonPageCommand, 
instructions are given to the state control unit 23 so as to 

. ■ ■ * 
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designate the page number specified by the operand of the 
navigation command as the page number X as well as to set the 

■ 

button number specified by the operand of the navigation command 
tothePSRIO (stepS58). Then, the state control unit 23 executes 
5 the "procedure when change is requested", with respect to the 
PSR 11 (step S59) . As mentioned above, the PSR 11 holds a value 
representing the currently presentedpage (i.e. the current page) . 
Thus, by performing the "procedure when change is requested" 
with respect to the PSR 11, the current page is determined. Then, 
10 the state control unit 23 performs the "procedure when playback 
condition is changed", with respect to the PSR10 (step S60) . 
This concludes the description of the flowchart shown in FIG. 
45. 

This concludes the description of the procedure for 
15 executing a navigation command. As stated above, a navigation 
command is executed when an associated button is activated. The 
button state changes are presented through the following 
procedure in response to a user operation (UO) . FIG. 4 6 shows 
the procedure for processing a UO. As shown in the flowchart, 
20 judgments are made in steps S61-S64 as to whether specific events 
have occurred. An occurrence of each event requires specific 
steps'to be performed. The processing then returns to the main 
routine. Specifically, in the step S61, a judgment is made as 
to whether the UO__mask_table field is set to "1" . If the field 
- 25 is set to "1", the processing returns to the main routine without 
performing any step. 

In the step S62, the judgment is made as to whether any 
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of the Move Up, Down, Left, and Right keys on the remote controller 
is pushed. At a push of any of the keys, the user_timeout timer 

18 and the selection^timeout timer 19 are reset (stepS6"5) . Then, 
another button is designated as the new current button (step 

5 S66) , followed by the judgment as to whether the auto_action_f lag 
of the newly designated current button is set to "01" (step S67) . 
If the auto_action_f lag is not set to "01", the processing returns 
to the main routine. On the other hand, if the auto_action_flag 
is set to "01", the current button (i) is activated (step S69) . 
10 Then, the variable animation (i) is set to "0" (step S70). 

In the step S63, the judgment is made as to whether a numeric 
input is made. If a numeric input is made, the user_timeout 
timer 18 and the selection_timeout timer 19 are reset (step S71) , 
and another button is designated as the new current button (step 
15 S72) . The processing then returns to the main routine. 

In the step S64 , a j udgment is made as .to whether an activate 
key on the remote controller is pushed. At a push of the activate 
key, the user_timeout timer 18 and the selection_timeout timer 

19 are reset (step S68), and then the current button (i) is 
20 .activated (step S69) . After the button state transition, the 

variable animation (i) is set to "0" (step S70) . Note that the 
step S66 of designating a new current button described above 
is performed by calling a subroutine shown in FIG. 47. Now, 
a description is given to the subroutine, with reference to the 
25 figure. b 

FIG . 4 7 is a flowchart of the current button change procedure . 
First, it is determined which of upper_button_id_ref , 
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lower_button_id_ref r lef t_button_id_ref , and 

right_button_id_ref fields within the neighbor_inf o associated 

* 

with the current button corresponds to the pushed key (stepS75) . 
Here^ let a button (Y) denote the current button,, and let 
5 a button X denote the button specified by the one of 
upper_button id ref , lower_button__id_ref , lef t_button_id ref, 
and right_button__id_ref fields (step S76). Then, the state 
control unit 23 performs the "procedure when change is requested'' 
with respect to the PSR 10 (step S77). As a result, the PSR 

10 10 is set to the value X. 

Thereafter, the variables animation (X) and animation (Y) 
are set to "0" (step S78), and returns to the main routirie. 

As described above, in the UO processing, a button is 
activated in response to a UO. Yet, the button state transition 

15 - to the activated state is made also when the 
selection_time_out_pts times out. Now, with reference to FIG. 
48, a description is given to the procedure for auto-activation 
of a button upon timeout of the selection_time_out_pts . 
FIG. 4 8 is a flowchart of the procedure for auto-activation 

20 of a button. First, a judgment is made as to whether the 
def ault__activated_button_id__ref field is set to M 00" or "FF"' 
(step S91) . If the field is set to "00", the processing returns 
to the main routine without performing any step. On the other 
hand, if the def ault_activated_button_id__ref is set to "FF", 

25^ the current button (i) is activated (step S93) . Thereafter, the 
variable animation (i) corresponding to the current button (i) 
is set to "0" (step S94), and returns to the main routine. 
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If the def ault_activated_button_id_ref field value is 
neither "00" nor "FF", the button specified by the 
def ault_activated_button__id_ref field value is determined to 
be the new current button (i) (step S92) . Thus, the button 
5 specified by the def ault__activated__button_id_jref field value 
receives the activated state (step S93) . Thereafter, the 
variable animation (i) corresponding to the current button (i) 
is set to "0" (step S94), and returns to the main routine. 

Through the above steps, the button in the selected state 
10 is automatically put into the activated state at an end of a 
predetermined duration. This concludes the description of the 
flowchart shown in FIG. 47. 

<Turn On/Off of Graphics Display in Pop-Up U/I> 
15 The description given above relates to the - procedure 

performed by the I-Graphics decoder 9 in the case where the 
user_inderf ace_model shows the Always-On U/I . In the case where 
the Pop-Up U/I is used, the I-Graphics decoder 9 operates as 
follows. First, similarly to the Always-On U/I case, the 
20 I-Graphics decoder 9 performs the decoding to obtain decoded 
graphics obj ects on the object buffer 35 . The I-Graphics decoder 
9 then renders the graphics objects to the interactive graphics 
plane 10 by the time shown by the PTS (DS n [ICS] ) . Up to this 
point, the I-Graphics decoder 9 operates in a similar manner 
25 to the case where Always-On U/I is used. Yet, the processing 
performed thereafter differs in that the graphics controller 
37 does not overlay the page stored on the interactive graphics 
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plane 10, and thus the page is not presented. That is to say, 
graphics controller 37 "turns off" the page by not outputting 

* 

the data stored on the interactive graphics plane 10 to the CLUT 
unit 15. With the page not presented on screen (i.e. turned 
off) , the graphics controller 37 waits for a user operation for 
Pop-Up__on. In response to a user operation for Pop-Up__on, the 

\ 

graphics controller 37 outputs the graphics data stored on the 
interactive graphics plane 10 to the CLUT unit 15 where the 
graphics data is overlaid with the video data. As a result, 
the page is "turned on" or presented on screen. As above, the 
graphics controller 37 turns on and off the graphics display 
to realize Pop-Up display. 

In the operations performed thereafter, there is no 
difference between the Always On U/I and the Pop-Up U/I . When 
any of the buttons on the main page is activated, a navigation 
command associated with the activated button is executed. When 
the select ion_timeout timer 19 times out, the current button 
on the main page is automatically activated and a sub page is 
presented on screen. 

When a user operation is made for Pop-Up_of f , the graphics 
controller 37 tunes off all on-screen pages, thereby leaving 
no page presented on screen. 

If no user operation is made for a certain period of time, 
the user__timeout timer 18 counts down toward zero. When the 
user_timeout timer 18 times out, the graphics controller 37 also 
turns off all on-screen pages, thereby leaving no page presented 
on screen. The turn on and turn off procedures described above 
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are performed to realize the state transition shown in FIG. 12A. 

As above, through the turn on and the turn off operations , 
Pop-Up U/I is implemented. 

As described above, the sixth embodiment of the present 
5 invention realizes the industrial production of playback 
apparatuses supporting the BD-ROM having the data structures 
as described in the first to fifth embodiments. This helps for 
widespread use of the BD-ROMs according to the first to fifth 
embodiments . 

10 

(SEVENTH EMBODIMENT) 

A seventh embodiment of the present invention relates to 
a manufacturing process of the BD-ROM. FIG. 49 is a flowchart 
of the manufacturing process of the BD-ROMs according to the 
15 first to fifth embodiments. 

The manufacturing process includes a material production 
step of recording video, audio, and the like (step S201) , an 
authoring step of creating an application format using an 
authoring device (step S202) , and a pressing step of creating 
20 an original master of the BD-ROM and performing stamping and 
bonding to complete the BD-ROM (step S203) . 

The BD-ROM authoring step includes the following steps 
S204-S213. 

Now, a description is given to the steps S204-S213 . In 
25 the step S240, control information, palette definition 
information, and graphics are generated. In the step S205, the 
control information, the palette definition information, and 
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the graphics are converted to functional segments. In the step 
S206, a PTS of each ICS is set based on the display timing of 

■ 

a picture to be synchronized with- In the step 3201 r a DTS [ODS] 
and a PTS [ODS] are set based on the PTS [ICS] . In the step S208, 
5 a DTS [ICS] and a PTS[PDS] are set based on the DTS [ODS]. 

In the step S209, changes in occupancy of each buffer in 
the player model are graphed. In the step S210, a judgment is 
made as to whether the graphed changes satisfy the constraints 
of the player model. If the judgment results in the negative, 

i 

10 the DTS and PTS of each functional segment are rewritten in the 
step S211 . If the judgment results in the affirmative, a graphics 
stream is generated in the step S212, and the graphics stream 

■i 

is multiplexed with a video stream and an audio stream, which 
are separately generated, to form an AV Clip in the step S213. 
15 After this, the AV Clip is adapted to the Blue-ray Disc Read-Only 
Format, to complete the application format. 

(MODIFICATIONS) 

Though the present invention has been described by way of 

20 the above embodiments, the present invention is not limited to 
those specific embodiments. The present invention may be 
embodied with any of the modifications (A) . to (L) below. The 
invention recited in each of the claims of this application 
includes extension and generalization of the above embodiments 

25 and their modifications below*. The degree of extension and 
generalization depends upon the state of the art in the technical 
field of the present invention at the time when the present 
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invention was made. 

4 

(A) The above embodiments describe the case where the BD-ROM 
is used as the recording medium. Main features of the present 

5 invention, however, lie in a graphics stream recorded on the 
recordingmedium, which does not rely on physical characteristics 

i 

of BD-ROMs . Therefore, the present invention is applicable to 
any recording medium that is capable of recording a graphics 
stream. Examples of such a recording medium include : an optical 

10 disc such as a DVD-ROM, a DVD-RAM, a~ DVD-RW, a DVD-R, a DVD+RW, 
a DVD+R, a CD-R, or a CD-RW; a magneto-optical disk such as a 
PD or an MO; a semiconductor memory card such as a CompactFlash 
card, a SmartMedia card, a Memory Stick card, a MultiMediaCard, 
or a PCMCIA card; amagnetic disk such as a flexible disk, SuperDisk, 

15 Zip, or Clik!; a removable hard disk drive such as ORB, Jaz, 
SparQ, SyJet, EZFley, or Microdrive, and a nonremovable hard 
disk drive. 

(B) The above embodiments describe the case where the 
20 playback apparatus decodes an AV Clip on the BD-ROM and outputs 

the decoded AV Clip to the television. As an alternative, the 
playback apparatus may be equipped with only a BD drive, with 
the remaining construction elements being provided in the 
television. In this case, the playback apparatus and the 
25 television can be incorporated in a home network connected with 
an IEEE 1394 connector. The above embodiments describe the case 
where the playback apparatus is connected to the television, 
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but the playback apparatus may instead be integrated with a 
display device. Also, the playback apparatus may include only 

* 

the system LSI (integrated circuit) which carries out an 
essential part of the processing. The playback apparatus and 
5 the integrated circuit are both an invention described in this 
specification . Accordingly, regardless of whether the playback 
apparatus or the integrated circuit is concerned, an act of 
manufacturing a playback apparatus based on the internal 
structure of the playback apparatus described in the sixth 

10 embodiment is an act of practicing the present invention. Also, 
any act of assigning with charge (i.e. for sale) or without charge 
(i.e. as a gift) , leasing, and importing the playback apparatus 
is an act of practicing the present invention. Likewise, an 
act of offering for assignment or lease of the reproduction 

15 apparatus using storefront displays, catalogs, or brochures is 
an act of practicing the present invention. 

(C) Information processing using the programs shown in the 
flowcharts is actually realized using hardware resources. 
,20 Accordingly, the programs which describe the procedure steps 
shown in the flowcharts are themselves an invention. The above 
embodiments all relate to the case where the programs are 
incorporated in the playback apparatus, but the programs can 
be used independently of the playback apparatus. Acts of 
* 25 practicing the programs include (1) an act of manufacturing, 
(2) an act of assigning with or without charge, (3) an act of 
leasing, (4) an act of importing, (5) an act of providing to 
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the public via a bi-directional ^-electronic communications 
network, and (6) an act of of fering for assignment or lease using 
storefront displays , catalogs, or brochures. 

(D) The time elements of the steps which are executed in 
a time series in each of the flowcharts can be regarded as the 
necessary elements of the present invention. This being so, 
a playback method shown by these flowcharts is an invention. 
If the processing shown in each flowchart is carried out by 
performing the steps in a time series so as to achieve the intended 
aim and the intended effect, it is considered as an act of 
practicing the playback method of the present invention. 

(E) When recording an AV Clip on the BD-ROM, an extension 
headermaybe addedto eachTSpacket in the AV Clip. The extension 
header is called a TP_extra__header , includes an 
arrival_time_stamp and a copy_permission_indicator, and has a 
data length of 4 bytes. TS packets with TP__extra_headers 
(hereafter "EX TS packets") are grouped in units of 32 packets, 
and each group is written to three sectors. One group made up 
of 32 EXTS packets has 6, 144 bytes ( = 32 x 192) , which is equivalent 
to a size of three sectors that is 6144 bytes (= 2048 x 3) . The 
32 EX TS packets contained in the three sectors are called an 
* Aligned Unit". 

In a home network connected with an IEEE 1394 connector, 
the playback apparatus transmits an Aligned Unit in the following 
manner. The reproduction apparatus removes a TP extra header 
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from each of the 32 EX TS packets in the Aligned Unit, encrypts 
the body of each TS packet according to the DTCP Specification, 
and outputs the encrypted TS packets.' When outputting the TS 
packets, the playback apparatus inserts an isochronous packet 
5 between adjacent TS packets. A position where the isochronous 
packet is inserted is based on a time shown by an 
arrival_time_stamp of the TP_extra_header. The playback 
apparatus outputs a DTCP__descriptor, as well as the TS packets. 
The DTCP_descriptor corresponds to a copy__permission_indicator 
10 in the TP_extraJieader . With the provision of the 
DTCP__descriptor indicating "copy prohibited", it is possible 
to prevent , when using the TS packets in the home network connected 
via the IEEE 1394 connector, the TS packets from being recorded 
to other devices. 

15 

(F) The above embodiments relate to the case where an AV 
Clip of the Blu-ray.Disc Read-Only Format is used as a digital 
stream, but the present invention may also be embodied with a 
VOB (Video Object) of the DVD-Video Format or the DVD-Video 

20 Recording Format. The VOB is a program stream that complies 
with the ISO/IEC 13818-1 Standard and is obtained by multiplexing 
a video stream and an audio stream. Also, the video stream in 
the AV Clip may be an MPEG4 video stream or a WMV video stream. 
Further, the audio stream in the AV Clip may be a Linear PCM 

25 audio stream, a Dolby AC-3 audio stream, an MP3 audio stream, 
an MPEG-AAC • audio stream, or a dts audio stream. 
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(G) The movie described in the above embodiments may be 
one obtained by encoding an analog image signal broadcast by 
analog broadcasting. Also, the movie may be stream data made 
up of a transport stream broadcast by digital broadcasting. 

Alternatively, an analog/digital image signal recorded on 
a videotape may be encoded to obtain content. Also, an 
analog/digital image signal directly captured by a video camera 
may be encoded to obtain a content. A digital work distributed 
by a distribution server- is applicable too. 

(H) Graphics objects described in the above embodiments 
is run-length encoded raster data. Run-length encoding is used 
for compression/encoding of graphics objects, because the 
run-length encoding is suitable for compression and 
decompression of subtitles. Subtitles have the property that 
a continuous length of the same pixel value in a horizontal 
direction is relatively long. Therefore, by performing 
compression using run-length encoding, a high compression rate 
can be attained. In addition, run-length encoding reduces a 
load for decompression, and is therefore suitable for realizing 
decoding by software. Nevertheless, the use of run-length 
encoding for graphics objects is not essential to the present 
invention. For example, graphics obj ects may be PNG data . Also,' 
graphics objects may be vector data instead of raster data. 
Further r graphics objects may be transparent patterns. 

(I) The transfer rate Rc may be determined so that clearing 
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and rendering of the graphics plane complete within a vertical 
blanking time. Suppose the vertical blanking time is 25% of 
1/29.93 seconds. Then Rc is determined to be 1 Gbps. By 
determining Rc in this way, graphics can be displayed smoothly, 
5 which achieves a significant effect on the practical use. 

Also, writing in sync with line scan can be used together 
with writing within a vertical blanking time. This ensures 
smooth presentation if the transfer rate Rc is 256 Mbps . 

10 (J) The above embodiments relate to the case where the 

playback apparatus is provided with the graphics plane. 
Alternatively, the playback apparatus may include a line buffer 
for storing uncompressed pixels of one line. Since conversion 
to -an image signal is performed for each horizontal row (line) , 

15 conversion to an image signal can equally be performed with the 
line buffer. 

(K) In the case of a merge point of multiple playback paths, 
a different button needs to be selected as default depending 

20 on a playback path having been taken. Thus, it is desirable 
to define such playback control in the static scenario that a 
value unique to each playback path is stored to a register of 
the playback apparatus at the time when the playback path is 
actually taken. The playback procedure may be set to put the 

25 button specified by the register value to the selected state. 
In this way, a different button is put to the selected state 
depending on a playback path having been taken. 
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(L) The graphics plane described in the sixth embodiment 

* 

preferably has a double buffer structure . If the graphics plane 
has a double buffer structure, even when large-sized graphics 
5 data needs to be rendered, which requires a time corresponding 
to several frames, each display composition may be 
instantaneously presented by switching between two buffers. 
Thus, a double buffer structure is effective when the menu of 
a full-screen size needs to be presented. 

10 

Industrial Applicability 

The playback apparatus according to the present invention 
is suitable for a personal use, for example in a home theater 
system. Since the above embodiments of the present invention 
15 disclose the internal structure, playback apparatuses having 
the internal structure can be produced on a large scale. Thus, 
the playback apparatus according to the present invention is 
industrially usable in its quality and has industrial 
applicability. 

20 
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CLAIMS 

4 

1. A recording medium having recorded thereon a video stream 
and a graphics stream, wherein: 

the video stream represents a moving picture made up of 
a plurality of pictures; 

the graphic stream is used for overlaying a multi-page menu 
on the moving picture, and includes interactive control 
information and graphics data; 

the graphics data is used for generating the multi-page 
menu; and 

the interactive control information includes time 
information used for controlling behavior of the multi-page menu 
in accordance with a playback proceeding of the video stream. 

2. The recording medium according to Claim 1, wherein: 

the multi-page menu includes a main page and a sub page; 
the multi-page menu behavior includes presentation of the 
main page in accordance with the playbackproceeding of the video 

< 

stream; and 

the time information includes timing information showing 
a time for presentation of the main page. 

3. The recording medium according to Claim 2, wherein: 

the multi-page menu behavior includes presentation of the 
sub page in accordance with the playback proceeding of the video 
stream; 
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the main page includes a button material for receiving a 
user operation instructing a playback apparatus to present the 
sub page; and 

the time information includes first timeout information 
5 showing a time for automatically activating the button material . 

4. The recording medium according to Claim 3, wherein: 

the interactive control information includes a navigation 
command for the playback apparatus to execute upon activation 
10 of the button material on the main page; and 

the navigation command includes a specification of a page 
of the multi-page menu to be presented and a specification of 
a button material to be presented in a selected state on the 
specified page. 

15 

5. The recording medium according to Claim 3, wherein: 

the multi-page menu behavior includes removal of the 
on-screen sub page in accordance with the playback proceeding 
of the video stream; and 
20 the time information includes second timeout information 

showing a time for automatically removing the on-screen sub page . 

6. The recording medium according to Claim 2 r wherein: 

the interactive control information is encapsulated in a 
25 packet; 

the time information is a presentation time stamp attached 
to the packet; and 
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a decoding time stamp attached to the packet shows a time 
for a playback apparatus to start decoding the interactive 
control information. 

5 7. The recording medium according to Claim 6, wherein: 

a value of the presentation time stamp is calculated by 

adding a sum of a first duration value and a second duration 

value to a value of the decoding time stamp; 

the first duration is a longer one of a duration taken to 
10 decode the graphics data included in the graphics stream and 

a duration taken to clear a graphics plane; and 

the second duration is a duration taken to transfer a 

complete set of uncompressed graphics used for presentation of 

the main page. 

15 

8. The recording medium according to Claim 7, wherein: 

the main page includes a plurality of button materials; 
a specific one of the button materials is presented in a 
selected state and the other button materials are presented in 
20 a normal state; 

the set of uncompressed graphics used for presentation of 
the main page includes 

uncompressed graphics obtained by decoding graphics 
data associated with the selected state of the specific button, 
25 and 

uncompressed graphics obtained by decoding graphics 
data associated with the normal state of each of the other buttons ; 
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and 

the transfer duration is a value calculated by dividing 
a total size of the set of uncompressed graphics by a transfer 
rate of the playback apparatus. 

5 

9. The recording medium according to Claim 7, wherein: 

the transfer duration is calculated by dividing a size T 
of the set of uncompressed graphics by a transfer rate of the 
playback apparatus; and 
10 the size T is calculated by a size TA + a size B - a size . 

NB, where 

the size TA is a to'tal size of graphics data associated 
with the normal state of each button included in the main page, 

the size B is a size of a button LRG, 
15 the size NB is a size of uncompressed graphics 

associated with the normal state of the button LRG, and 

the button LRG is a button material of which 
uncompressed graphics is largest in data size, out of all of 
the button materials included in the main page. 

10. The recording medium according to Claim 1, wherein: 

the interactive control information includes stream model 
information; 

the stream model information shows whether 
25 the graphics stream recorded on t-he recording medium 

is multiplexed with the video stream and to be supplied to a 
playback apparatus, or 
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the graphics stream recoded on the recording medium 
is non-multiplexed with the video stream and to be preloaded 

4 

to a memory of the playback apparatus; and 

the time information is valid only when the stream model 
information shows that the graphics stream is multiplexed with 
the video stream. 

11. The recording medium according to Claim 1, wherein: 

the multi-page menu includes a main page and a sub page; 

the behavior of the multi-page menu includes pop-up on 
presentation of the main page in accordance with the playback 
proceeding of the video stream; and 

the time information includes timing information showing 
a time at which the main page becomes available for the pop-up 
on presentation. 

12. The recording medium according to Claim 11, wherein: 

the multi-page menu behavior includes removable of 
on-screen pages presented by way of the pop-up on presentation; 
and 

the time information includes timeout information showing 
a time for automatically removing all on-screen pages of the 
multi-page menu. 

13. A playback apparatus 5 used for playing back a video stream 
and a graphics stream, comprising: 

a video decoder operable to decode the video stream to obtain 
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a moving picture; and 

a graphics decoder operable to decode the graphics stream 
so as to overlaying a multi-page menu on the moving picture, 
- wherein: 

5 the graphics stream includes interactive control 

information -and graphics data used to generate pages of the 
multi-page menu; and 

the graphics decoder includes 

a composition buffer operable to store the interactive 
10 control information, and 

a controller operable to control behavior of the 
multi-page menu in accordance with a playback proceeding of the 
video stream. 

15 14. The playback apparatus according to Claim 13,. wherein: 

the multi-page menu includes a main page and a sub page; 

and 

the controller controls the behavior of the multi-page menu 
so as to present the main page when playback of the video stream 
20 reaches a point corresponding to a time shown by a time stamp 
attached to a packet containing the interactive control 
information . 

15. The playback apparatus according to Claim 14, wherein: 
25 the main page includes a button material for receiving a 

user operation instructing to present the sub page; 

the playback apparatus further comprises a selection timer, 
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the selection timer being operable to be set to a value of a 
timeout shown by the time information upon presentation of the 
main page and to continuously count down while no user operation 
is received; and 
5 the controller controls the multi-page menu behavior to 

present the sub page by automatically activating the button 
material upon a timeout of the selection timer. 

16. The playback apparatus according to Claim 15 , wherein: 
10 the interactive control information includes a navigation 

command to be executed by the playback apparatus upon activation 
of a button material on the main page; and 

when a button material is activated, the controller decodes 
a navigation command associated with the activated button 
15 material to present a page specified by the navigation command 
and to put a button material specified by the navigation command 
to a selected state. 

17. The playback apparatus according to Claim 15, further 
20 comprising a user timeout timer, the user timeout timer being 

operable to be set to a value of a timeout shown by the time 
information upon presentation of the main page and to 
continuously count down while no user operation is received, 
■ wherein the controller automatically removes the on-screen 
25 sub page upon a timeout of the user timeout timer. 

18. The playback apparatus according to Claim 14, wherein: 
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the time stamp is a presentation time stamp attached to 
the packet; and 

the controller starts decoding the interactive control 
information at a time shown by a decoding time stamp attached 
5 to the packet. 

19. The playback apparatus according to Claim 18, further 
comprising a graphics plane operable to store graphics to be 
overlaid on the moving picture, wherein: 

10 the graphics decoder includes 

a processor operable to decode the graphics data 
included in the graphics stream, and 

an object buffer operable to store uncompressed 
graphics obtained by the processor; 
15 a value of the presentation time stamp is calculated by 

adding a sum of a first duration value and a second duration 
value to a value of the decoding time stamp; 

the first duration is a longer one of a duration taken for 
the graphics decoder to decode the graphics data included in 
20 the graphics stream and a duration taken for the graphics decoder 
- to clear the graphics plane; and 

the second duration is a time taken for the object buffer 
to transfer and render, to the graphics plane, a. complete set 
of uncompressed graphics necessary for presentation of the main 
25 page. 

* 

20. The playback apparatus according to Claim 19, wherein: 
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the main page includes a plurality of button materials; 
a specific one of the button materials is presented in a 

* 

selected state and the other button materials are presented in 
a normal state; 

5 the set of uncompressed graphics used for presentation of 

the main page includes 

uncompressed graphics obtained by decoding graphics 
data associated with the selected state of the specific button, 

and 

10 uncompressed graphics obtained by decoding graphics 

data associated with the normal state of each of the other buttons ; 
and 

the transfer duration is a value calculated by dividing 
a total size of the set of uncompressed graphics by a transfer 
15 rate at which uncompressed graphics are transferred from the 
object buffer to the graphics plane. 

21. The playback apparatus according to Claim 1-9, wherein: 
the transfer duration is calculated by dividing a size T 
20 of the set of uncompressed graphics by a transfer rate at which 

uncompressed graphics are transferred from the object buffer 

to the graphics plane; and 

the size T is calculated by a size TA + a size B - a size 

NB, where 

25 the size TA is a total size of graphics data associated 

with the normal state of each button included in the main page, 

the size B is a size of a button LRG, 
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the size NB is a 'size of uncompressed graphics 
associated with the normal state of the button LRG, and 

the button LRG is a button material of which 
uncompressed graphics is largest in data size, out of all of 
5 the button materials included in the main page. 

22. The playback apparatus according to Claim 13, wherein: 

the interactive control information stored in the 
composition buffer includes stream model information; 
10 when the stream model information shows that the graphics 

stream is multiplexed with the video stream, the controller 
discards the interactive control information in the composition 
buffer in accordance with a current playback point of the video 
* stream; and 

15 when the stream model information shows that the graphics 

stream is preloaded to the composition buffer, the controller 
retains the interactive control information regardless of the 
current playback point. 

20 23. The playback apparatus according to Claim 13, further 
comprising a graphics plane for storing pages of the multi-page 
menu one by one, wherein: 

the graphics decoder assembles a page on the graphics plane 
by decoding the graphics data included in the graphics stream 
25 and rendering uncompressed graphics obtained by the decoding 
to the graphics plane; and 

the controller turns on contents stored on the graphics 
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plane if a user operation for pop-up on is made after a current 
playback point reaches a predetermined point, so that the page 
is overlaid on the moving picture, and turns off the contents 
stored on the graphics plane if a user operation for pop-up off 
is made, so that the on-screen page is removed. 

24. The playback apparatus according to Claim 23, wherein 

the predetermined point is shown by a time stamp attached 
to a packet containing the interactive control information. 

25. The playback apparatus according to Claim 24, further 
' comprising a user timeout timer, the user timeout timer being 

operable to be set to a value of a timeout shown by the time 
information upon presentation of the main page and to 

continuously count down while no user operation is received, 

< 

wherein 

the controller automatically turns off the contents stored 
on the graphics plane to automatically remove all on-screen 
pages . 

26. A method for recording onto a recording medium, comprising: 

generating application data; and 

recording the application data onto the recording medium, 
wherein : 

the application data includes a video stream and a graphics 
stream; 

the video stream represents a moving picture made up of 
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a plurality of pictures; 

the graphic stream is used for overlaying a multi-page menu 
on the moving picture, and includes interactive control 
information and graphics data; 
5 the graphics data is used for generating the multi-page 

menu; and 

the interactive control information includes time 
information used for controlling behavior of the multi-page menu 
in accordance with a playback proceeding of the video stream. 

10 

27. A program used for enabling a computer to play back a video 
stream and a graphics stream, comprising: 

code operable to cause the computer to decode the. video 
stream to obtain a moving picture; and. 
15 code operable to cause the computer to decode the graphics 

stream so as to overlay a multi-page menu on the moving picture, 
wherein : 

the graphics stream includes interactive control 
information and graphics data used to generate pages of the 
20 multi-page menu; and 

the code operable to cause the computer to decode the video 
stream includes code operable to cause the computer to control 
behavior of the multi-page menu based on the interactive control 
information . 

25 

28 . A method for playing back a video stream and graphics stream, 
comprising: 
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decoding the video stream to obtain a moving picture; and 

decoding the graphics stream so as to overlay a multi-page 
menu on the -moving picture, wherein:- 

the graphics stream includes interactive control 
information and graphics data used to generate pages of the 
multi-page menu; and 

the decoding of the graphics stream includes controlling 
behavior of the multi-page menu based on the interactive control 
information. 
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